MingW: A lot of warnings + too slow compiling

hskoglund

Achilles' heel #1 in Linux/gcc: spaces in directories. Not in your build path, but in an include file:
Screenshot 2020-07-17 at 01.10.45.png

Edit: there's a quick-and-dirty solution (very short-term, will not survive a qmake), open that Makefile.Debug file in an editor and change line 1271 so that the problematic include directory is surrounded by double quotes (not single), i.e. so it looks like this extract:

...Desktop/rotortest/App -I"C:/Program Files/HDF_Group/HDF5/1.12.0/include" -IC:/Users/VM/Desktop/rotortest/App/thir...

Again: very short-term solution :-)

fem_dev

@hskoglund Wowwww!!! Thank you so much!!

I saw that this single quotes + blank spaces problem is very present in a lot of places of my Makefile.Debug.

@hskoglund said in MingW: A lot of warnings + too slow compiling:

Again: very short-term solution :-)

Is there a way to modify my rotortest.pro or any Qt Creator IDE configuration to resolve all occurrences at the same time?

Thank you so much @hskoglund ! You are the best!

Fixing this located problem, I will check if the compilation time is Ok or not.

fem_dev

@fem_dev said in MingW: A lot of warnings + too slow compiling:

I saw that this single quotes + blank spaces problem is very present in a lot of places of my Makefile.Debug.

I manually replaced all occurrences of this problem in the Makefile.debug and compile it using mingw32-make.exe.
It compiles ok! No warnings and no errors!!! Build and run with success!!

First problem: Fixed!!

Lets go to the final problem: The slower time to compile using MingW version.

Now, to measure how long each compiler takes to build my project, I have to be able to clean main build folder and build again switching between theses compilers inside the Qt Creator IDE:

MingW-w64 (GCC 10.1.0) (the last one)
MSVC 2019
GCC 9.3 (On my Ubuntu 20.04 machine - same hardware)

To do that, I need to be possible to clean up the build folder and rebuild all without fix the Makefile.debug manually.

Is there a way to config Qt Creator IDE (GCC Based) to generate only include lines with double quotes?

How can I do it?

hskoglund

Trying to add double quotes to those INCLUDEPATH += ... lines in the .pro file is almost impossible, because of the heavy munging that MSYS2 does (replaces all \ with / etc.)

But fear not, there's a sneaky way to get those double quotes to the command line:
in the .pro file, it involves replacing all the

INCLUDEPATH += "C:\Path\Path With Spaces"

with

QMAKE_CXXFLAGS += -I\"C:\Path\Path With Spaces\"

Note: you only need to this replacement for the INCLUDEPATH s that have spaces in them.

fem_dev

@fem_dev said in MingW: A lot of warnings + too slow compiling:

Lets go to the final problem: The slower time to compile using MingW version.

UPDATE 1

I move my HDF5 dependency libray folder to my HD root on C:\. So it fix the "single quotes" problem and Qt Creator IDE can build my project from scratch now and I can switch between the compilers.

Time to build my project:

MSYS MingW-w64 GCC 10.1.0 based: 70 seconds
MSVC 2019: 16 seconds
GCC 9.3: 13 seconds (On my Ubuntu 20.04 machine)

So, MSVC and GCC on Linux is very close time to build, but MSYS2 MingW-w64 is 5x slower using the same IDE version in the same machine.

Using:

Qt Creator IDE 4.12.4
Qt 5.15

My machines hardware are:

Intel Core i7 6 Cores 12 Threads 2.2GHz
32GB RAM
1TB NVMe storage

Is there a way to speed up the MSYS2 MingW-w64 compiler?

Obs.: Just to be clear, all 3 compilers are building my project without any warning or error messages!

fem_dev

@hskoglund said in MingW: A lot of warnings + too slow compiling:

QMAKE_CXXFLAGS += -I"C:\Path\Path With Spaces"

@hskoglund you are amazing!

hskoglund

Interesting about the compile times.
For another example, I have a project that used to compile in almost 1 hour on MSVC2015 I seem to remember.
Note: there's 2 large .h files that GitLab doesn't like, they can be downloaded here:
http://tripleboot.org/Projects/tctermscales.h.nogit
http://tripleboot.org/Projects/tcterms.h.nogit

I just tried now (for all tests: I nuked the build folder and the .pro.user file):

MSVC2019 on WIn10: 19 minutes 43 seconds but with error: TWTCData\tcterms.h.nogit(2826) : fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj

Mingw81_32 on Win10: 52 minutes 23 seconds

gcc 9.3.0 on Ubuntu 20.04: 52 minutes 23 seconds

Mingw81_32 on Wine on Ubuntu 20.04: 50 minutes and 54 seconds

MSYS2 Mingw32 on Win10: 1 minute and 54 seconds

Apple clang version 11.0.3 on MacOS 10.15.6: 36 seconds

So in my case, MSYS2 performs rather well (only to be beaten by clang).

@JKSH : so maybe you'll have to drink that beer, because my project actually builds slightly faster on Wine than on a real Windows. Admittedly not much of a difference though...

JKSH

@hskoglund said in MingW: A lot of warnings + too slow compiling:

@JKSH : so maybe you'll have to drink that beer, because my project actually builds slightly faster on Wine than on a real Windows. Admittedly not much of a difference though...

Dagnabbit! :-D

And I definitely wasn't expecting MinGW on Wine to beat native GCC on Ubuntu. Any ideas how that works?

@hskoglund said in MingW: A lot of warnings + too slow compiling:

gcc 9.3.0 on Ubuntu 20.04: 52 minutes 23 seconds
...
MSYS2 Mingw32 on Win10: 1 minute and 54 seconds

...or how this works?

hskoglund

Well the difference between Wine and native gcc on Ubuntu is only about 4% so it could be due to some caching in Wine.

But as to why MSYS2 mingw and clang just kills the others I don't know. The 2 big .h files have approx. 17000 QStringLiterals each (I get very nice loading/runtime performance using QStringLiterals)

Someday I should test with say 10000 QStringLiterals etc...

P.S. To get MSVC2019 to build, I added to the .pro file for MSVC2019:
win32-msvc: QMAKE_CXXFLAGS += "/bigobj"
and it just finished:
MSVC2019 on Win7: 18 minutes and 46 seconds. Pretty good!

J.Hilk

Since, mingw is a gnu port to windows, It's bound to be slower on Windows than on linux

When I changed ( about 2 years ago ) from mingw to MSVC as the default compiler, my project build time was slashed by a factor of 5, consistently across all projects!

May also be due to all the shady business and corners cut that MSVC does 🤷‍♂️.

Microsoft does not have a good track record with compilers and compiler safety :)

JonB

@J-Hilk said in MingW: A lot of warnings + too slow compiling:

from mingw to MSVC as the default compiler, my project build time was slashed by a factor of 5, consistently across all projects!

This is just depressing :( Why does it achieve that? They are compilers. "Cutting corners" could surely not get even vaguely near that. There must be some fundamental architectural difference. I've been meaning to raise a topic about this for a while, I note that MSVC makes heavy use of .pch files and gcc does not seem to do that at all?

jsulm

@JonB I guess Posix implementation on Windows is not that good...

fem_dev

@hskoglund said in MingW: A lot of warnings + too slow compiling:

MSYS2 Mingw32 on Win10: 1 minute and 54 seconds

Wowww! So fast!!
@hskoglund Only to confirm: this test was made using MSYS2 MingW-W64 or (32 bit version)?
I am confused about how your results can be just the reverse of mine. I would expect proportional results, as we are using the same compilers, on the same OS, etc ...

If you think there may be another configuration that I can change to try to get your MingW (Win10) build time, please let me know! I can try!

@hskoglund said in MingW: A lot of warnings + too slow compiling:

MSVC2019 on Win7: 18 minutes and 46 seconds. Pretty good!

With this last flag, MSVC2019 is pretty good too!

@J-Hilk said in MingW: A lot of warnings + too slow compiling:

by a factor of 5

Exactly. That's the build time difference that I have in my current project switching between theses 3 compilers.

@JonB said in MingW: A lot of warnings + too slow compiling:

MSVC makes heavy use of .pch files and gcc does not seem to do that at all?

a) Did you mean that MSVC create *.pch files automatically? Without an explicit developer setting?

b) Even though MSVC does this, in my tests above I always cleaned the build folder completely before recompiling. So there would be no *.pch file to optimize compilation time. Right?

JonB

@fem_dev
IIRC, MSVC .pch generation/usage requires a compiler flag. We used it on our projects, it made a big difference.

I assumed gcc did not have these, but it now appears I am incorrect --- https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html. I am surprised, then, since I have not a single example of a Qt gcc using them. When I would have thought it would be an ideal candidate. Then again, I see https://stackoverflow.com/questions/54727653/why-is-using-precompiled-headers-on-gcc-slower ....

It doesn't appear this is factor in your tests.

hskoglund

Hi, as I remember .pch files from my MFC days, they saved time when you compiled the same project over and over, but the tests in this topic are more "single-shot" flavored (i.e. you delete the build folder and .pro.user files every time). so they are immaterial here I think.

fem_dev

@hskoglund said in MingW: A lot of warnings + too slow compiling:

but the tests in this topic are more "single-shot" flavored (i.e. you delete the build folder and .pro.user files every time). so they are immaterial here I think.

Yes! That's it!

JonB

@fem_dev , @hskoglund
Yes, I acknowledged the .pch is not relevant to your tests.

FWIW, you might want to Google: why is mingw so slow, e.g. https://stackoverflow.com/questions/929495/why-is-mingw-very-slow. This is way beyond my experience, but I think it says if you're on a network (Active Directory) it behaves like a dog....! Admittedly that was over a decade ago, but who knows... :) Does https://groups.google.com/forum/#!topic/spatialite-users/PMi3sQxMAtA where it's "solved" help you?!

fem_dev

@JonB said in MingW: A lot of warnings + too slow compiling:

f you're on a network (Active Directory) i

Thank you @JonB , but my Windows machine is not in any network.
No special Windows configurations. Just a cleaner Windows 10 x64 installation with default configurations.

JonB

@fem_dev
The second post was the last resort (worth doing though!) switch any anti-virus off on Windows? That's my last thought. I am surprised by a factor as large as 5 times. Then again, @hskoglund's figures seem to vary by factor of 100x !! Surely there's something fishy here...? :)

fem_dev

@JonB said in MingW: A lot of warnings + too slow compiling:

switch any anti-virus off on Windows?

In this Windows machine, I don't have anti-virus or firewall...

@JonB said in MingW: A lot of warnings + too slow compiling:

vary by factor of 100x !!

Incredible!