4. Build with mingw32-make *very* slow

Build with mingw32-make *very* slow

  • Qt Champions 2018

    Hi all,

    I'm currently trying to compile a QMake subdirs project consisting of 10 .profiles with Qt 5.9.4 / MinGW 5.3 / Windows 7 x64 running as Guest in VirtualBox 5.2.6 (Guest additions are installed).

    My problem is, that it takes 10...11 seconds for mingw32-make to go through all the Makefiles and recognize that nothing is changed (and therefore nothing has to be build). I tried to use a ramdisk but that does not change anything.

    On my workstation running Windows 7 directly, the same takes approx. 3 seconds, which is acceptable.

    Any ideas or hints for this? Thanks in advance.

    14:20:05: Running steps for project xxxxControl...
14:20:05: Configuration unchanged, skipping qmake step.
14:20:05: Starting: "C:\Qt\Tools\mingw530_32\bin\mingw32-make.exe" -j 8
cd xxxxHAL\ && ( if not exist Makefile C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile R:\Daten\xxxxControl\xxxxHAL\xxxxHAL.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile 
mingw32-make[1]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL'
( if not exist Makefile.xxxxHalCanLibrary C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile.xxxxHalCanLibrary R:\Daten\xxxxControl\xxxxHAL\xxxxHalCanLibrary.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile.xxxxHalCanLibrary 
cd xxxxHalRemoteClient\ && ( if not exist Makefile C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile R:\Daten\xxxxControl\xxxxHAL\xxxxHalRemoteClient\xxxxHalRemoteClient.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile 
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL'
( if not exist Makefile.xxxxHalLibrary C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile.xxxxHalLibrary R:\Daten\xxxxControl\xxxxHAL\xxxxHalLibrary.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile.xxxxHalLibrary 
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteClient'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteClient'
cd xxxxHalRemoteTest\ && ( if not exist Makefile C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile R:\Daten\xxxxControl\xxxxHAL\xxxxHalRemoteTest\xxxxHalRemoteTest.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile 
cd xxxxHalRemoteSimple\ && ( if not exist Makefile C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile R:\Daten\xxxxControl\xxxxHAL\xxxxHalRemoteSimple\xxxxHalRemoteSimple.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile 
cd xxxxHalTerminal\ && ( if not exist Makefile C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile R:\Daten\xxxxControl\xxxxHAL\xxxxHalTerminal\xxxxHalTerminal.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile 
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL'
( if not exist Makefile.xxxxHalExample C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile.xxxxHalExample R:\Daten\xxxxControl\xxxxHAL\xxxxHalExample.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile.xxxxHalExample 
cd xxxxHalRemoteServer\ && ( if not exist Makefile C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile R:\Daten\xxxxControl\xxxxHAL\xxxxHalRemoteServer\xxxxHalRemoteServer.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile 
cd xxxxHalRemoteServer-direct\ && ( if not exist Makefile C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile R:\Daten\xxxxControl\xxxxHAL\xxxxHalRemoteServer-direct\xxxxHalRemoteServer-direct.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile 
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteSimple'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteSimple'
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalTerminal'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalTerminal'
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteTest'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteTest'
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL'
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteServer'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteServer'
mingw32-make[2]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteServer-direct'
mingw32-make[2]: Nothing to be done for 'first'.
mingw32-make[2]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL/xxxxHalRemoteServer-direct'
mingw32-make[1]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxHAL'
cd xxxxControl\ && ( if not exist Makefile C:\Qt\5.9.4\mingw53_32\bin\qmake.exe -o Makefile R:\Daten\xxxxControl\xxxxControl\xxxxControl.pro -spec win32-g++ "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Qt/Tools/mingw530_32/bin/mingw32-make -f Makefile 
mingw32-make[1]: Entering directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxControl'
mingw32-make[1]: Nothing to be done for 'first'.
mingw32-make[1]: Leaving directory 'R:/Daten/build-xxxxControl-Desktop_Qt_5_9_4_MinGW_32bit-Debug/xxxxControl'
14:20:15: The process "C:\Qt\Tools\mingw530_32\bin\mingw32-make.exe" exited normally.
14:20:15: Elapsed time: 00:11.```
    0 1 Reply
  • Moderators

    @aha_1980
    most probably this is a config issue?!
    How many (virtual) CPUs do you have configured?
    Did you enable CPU virtualization - if available (in BIOS settings)?
    ...

    0 1 Reply
  • Qt Champions 2018

    Hi @raven-worx,

    thanks for your answer.

    I just checked the BIOS settings (it's a Ryzen 7 1800X btw.):

    • PSS Support: Enabled (looks ACPI related)
    • NX Mode: Enabled (No-execute page protection)
    • SVM Mode: Enabled (CPU virtualization)
    • SMT Mode: Auto (simultaneous multithreading)
    • Core Leveling Mode: Automatic (number of cores per processor)

    and I assigned 8 cores to the VM.

    I don't mind that the compile speed is a bit slower, but the long turnaround times are quite annoying.

    0 1 Reply
  • Qt Champions 2018

    Update: I changed mingw32-make to jom in Creators build steps and that massively improved things.

    The check for changed files now takes under 1 second!

    I'm not sure if this setup is supported at all, but it looks promising.

    2 2 Replies

  • @aha_1980

    May be you have checked already, but are you sure that enabled multiple cores for MinGW-make as well?
    Respectively are you using parameter -j ?

    2
  • Qt Champions 2017

    @aha_1980 said in Build with mingw32-make *very* slow:

    I'm not sure if this setup is supported at all, but it looks promising.

    A tad late, but just for posterity:
    I'm pretty sure mingw uses single core (due to its own limitations), so you have to use jom to get all of your cores to build. From what I've experienced jom is working quite fine and I haven't encountered any issues with it and Qt. In the end this is the part after qmake's run and it's rather unrelated to Qt's internals.

    2
  • Qt Champions 2018

    Hi, and thanks for all your answers.

    @koahnig: Yes, I'm giving mingw32-make then -j 8 parameter to use my cores. But the slowness is already before compiling starts, when all the makefiles from my subdirs are checked. With mingw32-make you see the lines scrolling by and it seems to take a break at every 'Entering directory...' line. jom is just insanely fast.

    The effect is much less visible if you have only one .pro file and therefore only one Makefile.

    @kshegunov: Not too late at all :) Thanks, if you are going this way I'll try it too. I will see how it behaves if I'm working on some projects with it.

    0 2 Replies
  • Qt Champions 2018

    @aha_1980 Could it be related to your anti-virus software?

    1
  • Qt Champions 2018

    Hi @jsulm,

    no because there is no anti-virus ;) (these machines don't have physical internet connection so the risk is minimal)

    0
  • Qt Champions 2018

    I've marked this als solved, as I have a workaround for now. Thanks for all your suggestions!

    0

  • @aha_1980 said in Build with mingw32-make *very* slow:

    @koahnig: Yes, I'm giving mingw32-make then -j 8 parameter to use my cores. But the slowness is already before compiling starts, when all the makefiles from my subdirs are checked. With mingw32-make you see the lines scrolling by and it seems to take a break at every 'Entering directory...' line. jom is just insanely fast.

    That sounds interesting.
    Are you simply overwritting mingw32-make with jom in qt creator?

    0 1 Reply
  • Qt Champions 2017

    @koahnig said in Build with mingw32-make *very* slow:

    Are you simply overwritting mingw32-make with jom in qt creator?

    This is how I've done it in the past - going to the build settings and putting jom in "Override make" in the build steps (and clean steps), then adding -j4 to the make arguments text fields. There may be a better way, though.

    1
  • Qt Champions 2018

    Hi @koahnig

    Are you simply overwritting mingw32-make with jom in qt creator?

    Yes, that's what I do.

    There's also an old suggestion to have a jom checkbox for MinGW also. It was closed out of scope, but if there is enough interest maybe it can be reactived?

    In contrast to @kshegunov I don't specify any -j X to jom as it takes all cores by default.

    1

  • @aha_1980 @kshegunov

    Just to share my results.

    I have a couple of layered "subdirs" summing up to 18 projects.

    mingw32-make requires 25 seconds to check for a run when nothing has been changed.
    jom about 4 seconds for same.

    for rebuild the story is different
    mingw32-make requires about 4:59
    jom requires 6:11

    The clean is definitely faster with jom, but apparently it looses later on time when starting for the different projects.
    All checks are with compilation on ram disk with 8 cores

    jom was not happy with my environment settings MAKEFLAGS=-j8

    2 1 Reply
  • Qt Champions 2018

    @koahnig: thanks for sharing.

    The clean is definitely faster with jom, but apparently it looses later on time when starting for the different projects.
    All checks are with compilation on ram disk with 8 cores

    Interesting. As said, I'll do some more tests the next says and see how it behaves. For now jom is my winner.

    jom was not happy my environment settings MAKEFLAGS=-j8

    ... but it does not need it on the other hand ;)

    0
qmake 150 mingw 48 slow 12 subdirs 11
Log in to reply