Can't compile with jom in Windows - Makefile has bash syntax in it

morphine

Hi guys!

I've installed the Qt 5.11.2 sources and am looking to compile Qt in Windows. After running configure, I try and use jom to build Qt, but the process eventually fails.

Doing "jom /x err.txt" and looking at the output, I find this:

Error: syntax error in C:\Qt\tr-build\qtbase\qmake\Makefile line 44

jom: C:\Qt\tr-build\qtbase\qmake\Makefile.qmake-aux [C:\Qt\tr-build\qtbase\qmake\qmake.exe] Error 2
jom: C:\Qt\tr-build\qtbase\Makefile [sub-qmake-qmake-aux-pro-make_first] Error 2
jom: C:\Qt\tr-build\Makefile [module-qtbase-make_first] Error 2

I then proceed to open the Makefile in question and look for line 44, and lo and behold:

ifeq ($(SHELL), sh.exe)
    ifeq ($(wildcard ./sh.exe), )
        SH = 0
    else
        SH = 1
    endif
else
    SH = 1
endif

ifeq ($(SH), 1)
    RM_F = rm -f
    RM_RF = rm -rf
else
    RM_F = del /f
    RM_RF = rmdir /s /q
endif

Yep, that's not going to work well in a Windows environment. I've already double-checked that the configure script correctly uses the win32-g++ platform, but the end result is the same.

If I had to guess, it seems like there's a bug in the configure script? Or am I doing something else wrong? Thanks!

koahnig

@morphine

HI and welcome to devnet forum

Which archive did you download?
Fromwhere did you download?

For compilation on windows you should download the zip-file.

morphine

Hi there! I just used the Qt installer (Qt Maintenance Tool) and installed the 5.11.2 sources from there. I also tried using different MinGW environments but that didn't change much. I am, however, using jom from C:\Qt\Tools\Creator\Bin.

morphine

For what it's worth, I downloaded the sources package .zip separately, and also the jom utility as well. I got the same exact result.

aha_1980

@morphine which compiler do you want to use?

jom only works with MSVC

koahnig

@morphine

Jom has been introduced to do multi-thread compiling with MSVC compilers outside of the MSVC IDE. Therefore it is mainly used with Qt creator and MSVC compilers.

@aha_1980 said in Can't compile with jom in Windows - Makefile has bash syntax in it:

@morphine which compiler do you want to use?

jom only works with MSVC

Not exactly. You can even use it with MinGW. If I could handle the forum's search functionality better, I might be able to dig it out. Within the last year (?) there has been a thread on this topic. I believe 2 mods fellows where on the list. I had tried and it is possible.

aha_1980

@koahnig said in Can't compile with jom in Windows - Makefile has bash syntax in it:

Not exactly. You can even use it with MinGW. If I could handle the forum's search functionality better, I might be able to dig it out. Within the last year (?) there has been a thread on this topic. I believe 2 mods fellows where on the list. I had tried and it is possible.

I have tried it too, and yes, it is working (at least for small projects). However, it is not supported and therefore I cannot recommend using it. At least you are on your own if problems occur.

koahnig

@aha_1980 said in Can't compile with jom in Windows - Makefile has bash syntax in it:

@koahnig said in Can't compile with jom in Windows - Makefile has bash syntax in it:

Not exactly. You can even use it with MinGW. If I could handle the forum's search functionality better, I might be able to dig it out. Within the last year (?) there has been a thread on this topic. I believe 2 mods fellows where on the list. I had tried and it is possible.

I have tried it too, and yes, it is working (at least for small projects). However, it is not supported and therefore I cannot recommend using it. At least you are on your own if problems occur.

Agreed. The wording "only works" were a bit bold ;)
However as soon as a problem pops up, it is much better to hit the beaten path'

I have found the discussion https://forum.qt.io/topic/87909/build-with-mingw32-make-very-slow and my memory is still instact, since I thought you been in the discussion.

aha_1980

@koahnig

I have found the discussion https://forum.qt.io/topic/87909/build-with-mingw32-make-very-slow and my memory is still instact, since I thought you been in the discussion.

Nice, yes I started this thread. But let's not get to off-topic here.

So @morphine, do you want to use the MSVC compiler or MinGW?

At least for MSVC, you have to open a Visual Studio command prompt for building.

morphine

Thanks for the replies, everyone. In the meantime, I've tested MSVC + jom to compile, and that did in fact work (I was originally trying to use MinGW)

However, even though I'm not all that much into build processes, it would seem at first sight that the fault of the configure utility for creating a Makefile with bash syntax in it? Or was jom supposed to be able to interpret that anyway?

jsulm

@morphine You can use "-j NUMBER" as parameter to use more than one process to build using make:

make -j 8

morphine

This is relatively tangential, but is probably of interest to the crowd. For context, I'm using Qt compilation as a CPU benchmark. The configuration I have skips almost everything to keep the build time reasonable (i.e. "-no-dbus -no-open -skip this -skip that, etc etc")

After some informal tests on my own daily desktop, the build process (configure and make) with jom/MSVC takes around 28 minutes. Using MinGW 7.3 with a "-j 8" takes significantly longer, at around 39 minutes. That's almost a 40% difference. I did double-check that there were no errors and that a subsequent "make install"

I did verify that ming32-make was actually pulling in multiple jobs at the same time and kept an eye on Task Manager. (I've actually used make -j X before with an older Qt for the sampe exact purpose). From what I can gather, two things are different between jom/make.

The first one is that "configure" step with jom/MSVC seems to be multi-threaded itself, as it runs significantly faster compared to MinGW. That alone can easily count for a couple minutes' difference.

Then, the actual compilation with jom just seems to hit the CPU harder overall, for some reason, judging by the time delta and the machines's overall responsiveness while the compile job was running. For the record, Windows reports 100% CPU usage on both instances. I'll be running some tests on a dedicated box and report back.

aha_1980

@morphine

Just in case you wonder: It is long known that MSVC is the fastest compiler for Windows. gcc on Linux is in the same league, probably a bit faster (the Linux file system is really good for tasks like compiling). MinGW, which is gcc on Windows is significantly slower. I don't know how Clang behaves on Windows, but IIRC, some people have great hopes in it.

JonB

@aha_1980
Very interesting!

the Linux file system is really good for tasks like compiling

Excellent! Why?

morphine

@aha_1980, that's very useful information, thanks!

Other than having a benchmark that hits hard, another concern with this is ensuring it scales on many-core CPUs and the like. I'm guessing that'll depend as much on jom as it does on the compiler proper, but I should also probably MSVC to deliver on that front as well.

aha_1980

@JonB

Probably good caching? I don't really know.

I also have no experience with Mac, so I cannot compare in this direction.

morphine

For the record, a while back we (site I work for) actually benchmarked compilation on HDDs vs SSDs. While I'm sure that things have changed in the meantime, we found that for the actual build step, I/O played a very, very minor part.

mrjj

@morphine
Well using a ramdisk at work when compiling inside virtual machine
speed things up a lot. especially the linking part.
so while difference between spindisk (newer) and SSD might not
do much, upgrading to 6000/6000 MB/s ( ram disk speed)
did consistently work faster.
Its just a note. didn't really do much testing.

morphine

We did not, in fact, test that scenario. You've piqued my interest, maybe I will at some point :)

mrjj

@morphine
Well building Qt on ramdrive does take lots of ram :)
But real test could be fun. to see how much it matters. (for real)
for our project is just 20-25 secs. (out of 2 minutes) but
without proper testing it could be other factors.