MingW: A lot of warnings + too slow compiling

fem_dev · 17 Jul 2020, 02:10

@fem_dev said in MingW: A lot of warnings + too slow compiling:

I saw that this single quotes + blank spaces problem is very present in a lot of places of my Makefile.Debug.

I manually replaced all occurrences of this problem in the Makefile.debug and compile it using mingw32-make.exe.
It compiles ok! No warnings and no errors!!! Build and run with success!!

First problem: Fixed!!

Lets go to the final problem: The slower time to compile using MingW version.

Now, to measure how long each compiler takes to build my project, I have to be able to clean main build folder and build again switching between theses compilers inside the Qt Creator IDE:

MingW-w64 (GCC 10.1.0) (the last one)
MSVC 2019
GCC 9.3 (On my Ubuntu 20.04 machine - same hardware)

To do that, I need to be possible to clean up the build folder and rebuild all without fix the Makefile.debug manually.

Is there a way to config Qt Creator IDE (GCC Based) to generate only include lines with double quotes?

How can I do it?

hskoglund · 17 Jul 2020, 02:15

Trying to add double quotes to those INCLUDEPATH += ... lines in the .pro file is almost impossible, because of the heavy munging that MSYS2 does (replaces all \ with / etc.)

But fear not, there's a sneaky way to get those double quotes to the command line:
in the .pro file, it involves replacing all the

INCLUDEPATH += "C:\Path\Path With Spaces"

with

QMAKE_CXXFLAGS += -I\"C:\Path\Path With Spaces\"

Note: you only need to this replacement for the INCLUDEPATH s that have spaces in them.

fem_dev · wrote on 17 Jul 2020, 02:10

@fem_dev said in MingW: A lot of warnings + too slow compiling:

Lets go to the final problem: The slower time to compile using MingW version.

UPDATE 1

I move my HDF5 dependency libray folder to my HD root on C:\. So it fix the "single quotes" problem and Qt Creator IDE can build my project from scratch now and I can switch between the compilers.

Time to build my project:

MSYS MingW-w64 GCC 10.1.0 based: 70 seconds
MSVC 2019: 16 seconds
GCC 9.3: 13 seconds (On my Ubuntu 20.04 machine)

So, MSVC and GCC on Linux is very close time to build, but MSYS2 MingW-w64 is 5x slower using the same IDE version in the same machine.

Using:

Qt Creator IDE 4.12.4
Qt 5.15

My machines hardware are:

Intel Core i7 6 Cores 12 Threads 2.2GHz
32GB RAM
1TB NVMe storage

Is there a way to speed up the MSYS2 MingW-w64 compiler?

Obs.: Just to be clear, all 3 compilers are building my project without any warning or error messages!

fem_dev · wrote on 17 Jul 2020, 02:15

@hskoglund said in MingW: A lot of warnings + too slow compiling:

QMAKE_CXXFLAGS += -I"C:\Path\Path With Spaces"

@hskoglund you are amazing!

hskoglund · 17 Jul 2020, 03:00

Interesting about the compile times.
For another example, I have a project that used to compile in almost 1 hour on MSVC2015 I seem to remember.
Note: there's 2 large .h files that GitLab doesn't like, they can be downloaded here:
http://tripleboot.org/Projects/tctermscales.h.nogit
http://tripleboot.org/Projects/tcterms.h.nogit

I just tried now (for all tests: I nuked the build folder and the .pro.user file):

MSVC2019 on WIn10: 19 minutes 43 seconds but with error: TWTCData\tcterms.h.nogit(2826) : fatal error C1128: number of sections exceeded object file format limit: compile with /bigobj

Mingw81_32 on Win10: 52 minutes 23 seconds

gcc 9.3.0 on Ubuntu 20.04: 52 minutes 23 seconds

Mingw81_32 on Wine on Ubuntu 20.04: 50 minutes and 54 seconds

MSYS2 Mingw32 on Win10: 1 minute and 54 seconds

Apple clang version 11.0.3 on MacOS 10.15.6: 36 seconds

So in my case, MSYS2 performs rather well (only to be beaten by clang).

@JKSH : so maybe you'll have to drink that beer, because my project actually builds slightly faster on Wine than on a real Windows. Admittedly not much of a difference though...

JKSH · wrote on 17 Jul 2020, 03:00

@hskoglund said in MingW: A lot of warnings + too slow compiling:

@JKSH : so maybe you'll have to drink that beer, because my project actually builds slightly faster on Wine than on a real Windows. Admittedly not much of a difference though...

Dagnabbit! :-D

And I definitely wasn't expecting MinGW on Wine to beat native GCC on Ubuntu. Any ideas how that works?

@hskoglund said in MingW: A lot of warnings + too slow compiling:

gcc 9.3.0 on Ubuntu 20.04: 52 minutes 23 seconds
...
MSYS2 Mingw32 on Win10: 1 minute and 54 seconds

...or how this works?

hskoglund · wrote on 17 Jul 2020, 03:12

Well the difference between Wine and native gcc on Ubuntu is only about 4% so it could be due to some caching in Wine.

But as to why MSYS2 mingw and clang just kills the others I don't know. The 2 big .h files have approx. 17000 QStringLiterals each (I get very nice loading/runtime performance using QStringLiterals)

Someday I should test with say 10000 QStringLiterals etc...

P.S. To get MSVC2019 to build, I added to the .pro file for MSVC2019:
win32-msvc: QMAKE_CXXFLAGS += "/bigobj"
and it just finished:
MSVC2019 on Win7: 18 minutes and 46 seconds. Pretty good!

J.Hilk · 17 Jul 2020, 07:06

Since, mingw is a gnu port to windows, It's bound to be slower on Windows than on linux

When I changed ( about 2 years ago ) from mingw to MSVC as the default compiler, my project build time was slashed by a factor of 5, consistently across all projects!

May also be due to all the shady business and corners cut that MSVC does 🤷‍♂️.

Microsoft does not have a good track record with compilers and compiler safety :)

JonB · 17 Jul 2020, 07:11

@J-Hilk said in MingW: A lot of warnings + too slow compiling:

from mingw to MSVC as the default compiler, my project build time was slashed by a factor of 5, consistently across all projects!

This is just depressing :( Why does it achieve that? They are compilers. "Cutting corners" could surely not get even vaguely near that. There must be some fundamental architectural difference. I've been meaning to raise a topic about this for a while, I note that MSVC makes heavy use of .pch files and gcc does not seem to do that at all?

jsulm · J JonB 17 Jul 2020, 07:06

@JonB I guess Posix implementation on Windows is not that good...

fem_dev · 17 Jul 2020, 14:46

@hskoglund said in MingW: A lot of warnings + too slow compiling:

MSYS2 Mingw32 on Win10: 1 minute and 54 seconds

Wowww! So fast!!
@hskoglund Only to confirm: this test was made using MSYS2 MingW-W64 or (32 bit version)?
I am confused about how your results can be just the reverse of mine. I would expect proportional results, as we are using the same compilers, on the same OS, etc ...

If you think there may be another configuration that I can change to try to get your MingW (Win10) build time, please let me know! I can try!

@hskoglund said in MingW: A lot of warnings + too slow compiling:

MSVC2019 on Win7: 18 minutes and 46 seconds. Pretty good!

With this last flag, MSVC2019 is pretty good too!

@J-Hilk said in MingW: A lot of warnings + too slow compiling:

by a factor of 5

Exactly. That's the build time difference that I have in my current project switching between theses 3 compilers.

@JonB said in MingW: A lot of warnings + too slow compiling:

MSVC makes heavy use of .pch files and gcc does not seem to do that at all?

a) Did you mean that MSVC create *.pch files automatically? Without an explicit developer setting?

b) Even though MSVC does this, in my tests above I always cleaned the build folder completely before recompiling. So there would be no *.pch file to optimize compilation time. Right?

JonB · wrote on 17 Jul 2020, 14:46

@fem_dev
IIRC, MSVC .pch generation/usage requires a compiler flag. We used it on our projects, it made a big difference.

I assumed gcc did not have these, but it now appears I am incorrect --- https://gcc.gnu.org/onlinedocs/gcc/Precompiled-Headers.html. I am surprised, then, since I have not a single example of a Qt gcc using them. When I would have thought it would be an ideal candidate. Then again, I see https://stackoverflow.com/questions/54727653/why-is-using-precompiled-headers-on-gcc-slower ....

It doesn't appear this is factor in your tests.

hskoglund · 17 Jul 2020, 15:00

Hi, as I remember .pch files from my MFC days, they saved time when you compiled the same project over and over, but the tests in this topic are more "single-shot" flavored (i.e. you delete the build folder and .pro.user files every time). so they are immaterial here I think.

fem_dev · 17 Jul 2020, 15:15

@hskoglund said in MingW: A lot of warnings + too slow compiling:

but the tests in this topic are more "single-shot" flavored (i.e. you delete the build folder and .pro.user files every time). so they are immaterial here I think.

Yes! That's it!

JonB · 17 Jul 2020, 17:03

@fem_dev , @hskoglund
Yes, I acknowledged the .pch is not relevant to your tests.

FWIW, you might want to Google: why is mingw so slow, e.g. https://stackoverflow.com/questions/929495/why-is-mingw-very-slow. This is way beyond my experience, but I think it says if you're on a network (Active Directory) it behaves like a dog....! Admittedly that was over a decade ago, but who knows... :) Does https://groups.google.com/forum/#!topic/spatialite-users/PMi3sQxMAtA where it's "solved" help you?!

fem_dev · 17 Jul 2020, 17:24

@JonB said in MingW: A lot of warnings + too slow compiling:

f you're on a network (Active Directory) i

Thank you @JonB , but my Windows machine is not in any network.
No special Windows configurations. Just a cleaner Windows 10 x64 installation with default configurations.

JonB · 17 Jul 2020, 22:15

@fem_dev
The second post was the last resort (worth doing though!) switch any anti-virus off on Windows? That's my last thought. I am surprised by a factor as large as 5 times. Then again, @hskoglund's figures seem to vary by factor of 100x !! Surely there's something fishy here...? :)

fem_dev · J JonB 17 Jul 2020, 17:24

@JonB said in MingW: A lot of warnings + too slow compiling:

switch any anti-virus off on Windows?

In this Windows machine, I don't have anti-virus or firewall...

@JonB said in MingW: A lot of warnings + too slow compiling:

vary by factor of 100x !!

Incredible!