C++ program runs much slower on Windows vs Linux when doing file operations?

Question

I use an open source project called ChatScript for natural language processing app development.

When you execute a build operation with ChatScript, it scans all the script files that comprise your chat-bot. In my case, that's hundreds of files. This process takes nearly 30 times longer on Windows 8.1 than it does on Ubuntu 16.04. Therefore I do use Linux for much of my work, but there is a part of my work that I have to do on Windows because of certain associated tools, so I would like to modify the code base so that Windows ChatScript compiles are as fast as on Linux.

Can anyone think of a reason the code would run so much slower on Windows vs. Linux? Are there some C++ file operation codes (read/write/etc.) that are known to be much slower on Windows compared to Linux due to variances in the C++ run-time libraries running on each platform?

Answer 1

By "code running slow" in your last paragraph I'm assuming from context that you're referring to the compiler???

I've encountered frequently and consistently over many years a general, significant performance difference between linux and Windows in disk I/O. NTFS (Windows file system) and the linux file systems handle the situation of lots of files differently, and linux is always quicker in the circumstances I've been in.

You may benefit from some of the pointers in answers to questions like How do I get Windows to go as fast as Linux for compiling C++?, like defragmenting your windows drive, and checking how the compiler optimisations are configured; some of them can slow down the compiler (although an aggressive compiler optimisation setting can slow the compiler down, you produce a faster executable at the end, but that might be something you switch to after most of your development is done).

But doing all those things for me has never got the Windows compile to be quicker than on linux using equivalent disk hardware, not once. If your code is on the one disk and sourced for both compiles, any improvement you'll see in the Windows build (e.g. because the code's put on an SSD) will likely be replicated in an improvement in the linux build as well.

Answer 2

Just to confirm I found the same thing. Ran the same Chatscript scripts on an average Mac and on a fast XPS 15. The Mac compiled the code 30-50 times faster than Windows. What's weird is ChatScript was originally developed for Windows. I also have not worked out why such a gigantic lag, in spite of the Windows PC hardware being much more powerful than the Mac running the script.

Answer 3

So I have come across both an explanation and a (partial) solution. There are two areas of lag in Windows compared to Linux:

• Networking behaviour. According to the creator of ChatScript, Bruce Wilcox, the Windows server code is worse under the hood, and is also implemented worse in ChatScript for Windows vs ChatScript for Linux. The lag here is however minor.

• Build times. This is where building the bot in Linux takes 10-20 seconds and on Windows 4-5 minutes. Turns out that the reason is the antivirus:

"Curious about the huge discrepancy, one of our hardware engineers
did some profiling and found the real culprit to be the anti-virus
software. Disabling the real-time virus protection feature of
Windows Defender brought the 4 minutes down to 14 seconds! Even
keeping Windows Defender active, but excluding the
ChatScript-master folder solved much of the slowdown problem,
resulting in about 20 s for :build 0 to complete." (https://www.chatbots.org/ai_zone/viewthread/3575/)

So for OP, if you exclude CS from Windows Defender, or switch it off, the build differential will largely disappear.