Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Can't compile with jom in Windows - Makefile has bash syntax in it



  • Hi guys!

    I've installed the Qt 5.11.2 sources and am looking to compile Qt in Windows. After running configure, I try and use jom to build Qt, but the process eventually fails.

    Doing "jom /x err.txt" and looking at the output, I find this:

    Error: syntax error in C:\Qt\tr-build\qtbase\qmake\Makefile line 44
    
    jom: C:\Qt\tr-build\qtbase\qmake\Makefile.qmake-aux [C:\Qt\tr-build\qtbase\qmake\qmake.exe] Error 2
    jom: C:\Qt\tr-build\qtbase\Makefile [sub-qmake-qmake-aux-pro-make_first] Error 2
    jom: C:\Qt\tr-build\Makefile [module-qtbase-make_first] Error 2
    

    I then proceed to open the Makefile in question and look for line 44, and lo and behold:

    ifeq ($(SHELL), sh.exe)
        ifeq ($(wildcard ./sh.exe), )
            SH = 0
        else
            SH = 1
        endif
    else
        SH = 1
    endif
    
    ifeq ($(SH), 1)
        RM_F = rm -f
        RM_RF = rm -rf
    else
        RM_F = del /f
        RM_RF = rmdir /s /q
    endif
    

    Yep, that's not going to work well in a Windows environment. I've already double-checked that the configure script correctly uses the win32-g++ platform, but the end result is the same.

    If I had to guess, it seems like there's a bug in the configure script? Or am I doing something else wrong? Thanks!



  • @morphine

    HI and welcome to devnet forum

    Which archive did you download?
    Fromwhere did you download?

    For compilation on windows you should download the zip-file.



  • Hi there! I just used the Qt installer (Qt Maintenance Tool) and installed the 5.11.2 sources from there. I also tried using different MinGW environments but that didn't change much. I am, however, using jom from C:\Qt\Tools\Creator\Bin.



  • For what it's worth, I downloaded the sources package .zip separately, and also the jom utility as well. I got the same exact result.


  • Lifetime Qt Champion

    @morphine which compiler do you want to use?

    jom only works with MSVC



  • @morphine

    Jom has been introduced to do multi-thread compiling with MSVC compilers outside of the MSVC IDE. Therefore it is mainly used with Qt creator and MSVC compilers.

    @aha_1980 said in Can't compile with jom in Windows - Makefile has bash syntax in it:

    @morphine which compiler do you want to use?

    jom only works with MSVC

    Not exactly. You can even use it with MinGW. If I could handle the forum's search functionality better, I might be able to dig it out. Within the last year (?) there has been a thread on this topic. I believe 2 mods fellows where on the list. I had tried and it is possible.


  • Lifetime Qt Champion

    @koahnig said in Can't compile with jom in Windows - Makefile has bash syntax in it:

    Not exactly. You can even use it with MinGW. If I could handle the forum's search functionality better, I might be able to dig it out. Within the last year (?) there has been a thread on this topic. I believe 2 mods fellows where on the list. I had tried and it is possible.

    I have tried it too, and yes, it is working (at least for small projects). However, it is not supported and therefore I cannot recommend using it. At least you are on your own if problems occur.



  • @aha_1980 said in Can't compile with jom in Windows - Makefile has bash syntax in it:

    @koahnig said in Can't compile with jom in Windows - Makefile has bash syntax in it:

    Not exactly. You can even use it with MinGW. If I could handle the forum's search functionality better, I might be able to dig it out. Within the last year (?) there has been a thread on this topic. I believe 2 mods fellows where on the list. I had tried and it is possible.

    I have tried it too, and yes, it is working (at least for small projects). However, it is not supported and therefore I cannot recommend using it. At least you are on your own if problems occur.

    Agreed. The wording "only works" were a bit bold ;)
    However as soon as a problem pops up, it is much better to hit the beaten path'

    I have found the discussion https://forum.qt.io/topic/87909/build-with-mingw32-make-very-slow and my memory is still instact, since I thought you been in the discussion.


  • Lifetime Qt Champion

    @koahnig

    I have found the discussion https://forum.qt.io/topic/87909/build-with-mingw32-make-very-slow and my memory is still instact, since I thought you been in the discussion.

    Nice, yes I started this thread. But let's not get to off-topic here.

    So @morphine, do you want to use the MSVC compiler or MinGW?

    At least for MSVC, you have to open a Visual Studio command prompt for building.



  • Thanks for the replies, everyone. In the meantime, I've tested MSVC + jom to compile, and that did in fact work (I was originally trying to use MinGW)

    However, even though I'm not all that much into build processes, it would seem at first sight that the fault of the configure utility for creating a Makefile with bash syntax in it? Or was jom supposed to be able to interpret that anyway?


  • Lifetime Qt Champion

    @morphine You can use "-j NUMBER" as parameter to use more than one process to build using make:

    make -j 8
    


  • This is relatively tangential, but is probably of interest to the crowd. For context, I'm using Qt compilation as a CPU benchmark. The configuration I have skips almost everything to keep the build time reasonable (i.e. "-no-dbus -no-open -skip this -skip that, etc etc")

    After some informal tests on my own daily desktop, the build process (configure and make) with jom/MSVC takes around 28 minutes. Using MinGW 7.3 with a "-j 8" takes significantly longer, at around 39 minutes. That's almost a 40% difference. I did double-check that there were no errors and that a subsequent "make install"

    I did verify that ming32-make was actually pulling in multiple jobs at the same time and kept an eye on Task Manager. (I've actually used make -j X before with an older Qt for the sampe exact purpose). From what I can gather, two things are different between jom/make.

    The first one is that "configure" step with jom/MSVC seems to be multi-threaded itself, as it runs significantly faster compared to MinGW. That alone can easily count for a couple minutes' difference.

    Then, the actual compilation with jom just seems to hit the CPU harder overall, for some reason, judging by the time delta and the machines's overall responsiveness while the compile job was running. For the record, Windows reports 100% CPU usage on both instances. I'll be running some tests on a dedicated box and report back.


  • Lifetime Qt Champion

    @morphine

    Just in case you wonder: It is long known that MSVC is the fastest compiler for Windows. gcc on Linux is in the same league, probably a bit faster (the Linux file system is really good for tasks like compiling). MinGW, which is gcc on Windows is significantly slower. I don't know how Clang behaves on Windows, but IIRC, some people have great hopes in it.



  • @aha_1980
    Very interesting!

    the Linux file system is really good for tasks like compiling

    Excellent! Why?



  • @aha_1980, that's very useful information, thanks!

    Other than having a benchmark that hits hard, another concern with this is ensuring it scales on many-core CPUs and the like. I'm guessing that'll depend as much on jom as it does on the compiler proper, but I should also probably MSVC to deliver on that front as well.


  • Lifetime Qt Champion

    @JonB

    Probably good caching? I don't really know.

    I also have no experience with Mac, so I cannot compare in this direction.



  • For the record, a while back we (site I work for) actually benchmarked compilation on HDDs vs SSDs. While I'm sure that things have changed in the meantime, we found that for the actual build step, I/O played a very, very minor part.


  • Lifetime Qt Champion

    @morphine
    Well using a ramdisk at work when compiling inside virtual machine
    speed things up a lot. especially the linking part.
    so while difference between spindisk (newer) and SSD might not
    do much, upgrading to 6000/6000 MB/s ( ram disk speed)
    did consistently work faster.
    Its just a note. didn't really do much testing.



  • We did not, in fact, test that scenario. You've piqued my interest, maybe I will at some point :)


  • Lifetime Qt Champion

    @morphine
    Well building Qt on ramdrive does take lots of ram :)
    But real test could be fun. to see how much it matters. (for real)
    for our project is just 20-25 secs. (out of 2 minutes) but
    without proper testing it could be other factors.



  • @mrjj

    Well using a ramdisk at work when compiling inside virtual machine

    Now what is that "ramdisk" exactly? My Windows hosts Linux in a VirtualBox VM, are you saying I could have some sort of ramdisk in the Linux VM?


  • Lifetime Qt Champion

    @JonB
    well its a block device in ram. like a normal disk.
    and yes inside the wm.
    https://www.jamescoyle.net/how-to/943-create-a-ram-disk-in-linux

    on windows i use
    Softperfect ram disk.



  • @morphine said in Can't compile with jom in Windows - Makefile has bash syntax in it:

    @aha_1980, that's very useful information, thanks!

    Other than having a benchmark that hits hard, another concern with this is ensuring it scales on many-core CPUs and the like. I'm guessing that'll depend as much on jom as it does on the compiler proper, but I should also probably MSVC to deliver on that front as well.

    When using Qt creator you require jom for MSVC for parallel compilation. IIRC it compensates the lack of nmake to do parallel compilations on its own. This is done by the MSVC IDE there.

    Qt creator and MinGW does allow parallel compilation on windows. You need to use the -j parameter or environment settings. There no need to use jom with MinGW, but you can (see the link I had posted above for CPU comparison).

    Also RAM disks help to speed up. I place my shadow on the ram disk.
    There was already a discussion involving @mrjj https://forum.qt.io/topic/80933/qtcreator-shadow-build-directory-structure/19
    However, I thought that there were some numbers given in comparison.


Log in to reply