Android Project - Make.exe parallel jobs



  • Hello everybody!

    I'm currently working on a Qt project targeting Android and I noticed a really strange behavior: if I add a "***-j***" argument to the Make build step, the compilation (obviously) ends up requiring too much resources (in particular RAM) so I tried to change it to "***-j4***" or any "***-jn***" but it does not work, it spawns only a single process.

    I investigated a little bit and saw that someone complains about the MinGW version of Make provided by the Android NDK. To be sure I made a quick test creating a Makefile calling two children Makefiles underneath and launching it with the same Make version and it works perfectly.

    I suppose then that the issue is not related to Make.exe but probably something else; initially I thought to Qt Creator as the guilt but launching the build from command line reproduces the same issue.

    Could it be something related to QMake?

    My build command is

    qmake.exe "C:\Projects\MyProject\project.pro" -r -spec android-clang "QMAKE_PROJECT_DEPTH=0" "CONFIG+=debug" -after DEFINES+=BLM_APP_BUILD_VERSION=\\\\\\\"\"000\\\\\\\"\" && C:\Tools\android\android-ndk-r18b\prebuilt\windows-x86_64\bin\make.exe -j 100
    

    Details about the environment:

    • Project target: Android ARMv7
    • NDK: r18b
    • Qt version: 5.12.1
    • Qt Creator version: 4.9.2
    • Host operating system: Windows 10

    Any suggestion? I don't see why "***-j***" should work while "***-jn***" doesn't...

    Thanks a lot!



  • @paolot

    Hi and welcome to devnet forum

    Apparently you are starting qmake outside of Qt creator. How do you that?
    You should be able to check the actual make command being carried out. Alternatively, you can start the make in the command separately to ensure that qmake has no influence which I doubt anyway.



  • Hi, thanks for replying.

    Looking at the "Compile Output" tab I see the following when I start the build:

    13:08:39: Running steps for project stpalm...
    13:08:39: Configuration unchanged, skipping qmake step.
    13:08:39: Starting: "C:\Tools\android\android-ndk-r18b\prebuilt\windows-x86_64\bin\make.exe" -j12
    cd utility\ && ( if not exist Makefile C:\Qt\5.12.1\android_armv7\bin\qmake.exe -o Makefile C:\Projects\MyProject\utility\utility.pro -spec android-clang "CONFIG+=debug" "CONFIG+=qml_debug" ) && C:/Tools/android/android-ndk-r18b/prebuilt/windows-x86_64/bin/make -f Makefile 
    

    As you can see this is just the first step since my project consists of several subfolders (one of which is "utility"), so the make command is run once for each subfolder.

    The flag "***-j12***" is applied but does not work at all unfortunately.



  • @paolot

    Do you define a sequence of those subfolders?
    If yes, it will follow the order you are giving. E.g. if subfolder3 depends on subfolder2, subfolder 2 depends on subfolder 1 and subfolder1 depends on subfolder, you cannot expect those subfolders re compiled in parallel. I have a similar structure and the subfolders will be processed in sequence.

    Otherwise this is anyway something up to the make you are using. qmake is preparing only the different make files for processing by make.



  • I know that dependencies between projects have to be processed sequentially, in fact I'm talking about parallel build within the single project. It is working if I use just "-j" without specifying the number of jobs, so I would expect that it worked similarly even with "-j".

    I already reached the Android NDK forum and they think that the issue could be related to QMake (obviously), but the most reasonable argument right now is that, as you already said, QMake just prepares the Makefiles, any issue with the parallel building should be caused by Make itself.

    I'll try to find an example project to build on my machine in order to reproduce the issue, so that I can share with you and with the NDK developers.

    Thanks a lot for the moment!



  • @paolot

    How do you set in creator the flag?

    In creator on building settings page you can add -j12 or whatever you like that shall be used resp. in my creator 4.9.2 there is now the possibility to set the number of parallel jobs and it says 8. Never entered this number probably taken out of my environment.

    IIRC in the past I did not want to add the flag per project/application. Therefore I have added in my windows environment

    MAKEFLAGS=-j8
    

    That may explain also why creator is recognizing the number of parallel jobs.

    As it looks in your last post make is not started with an additional command line argument for parallel jobs.



  • @koahnig

    Sorry, I made some confusion while writing the last post, what I meant is that:

    • Using "-j" to build works properly but it takes to many system resources
    • Using "-j12" or any other number does not work at all, using any number of jobs makes the build process to use just one job

    I made some new tests that might be useful to understand what's going on under the hood. I made a full compilation from Qt Creator (running both QMake and Make), then I opened a command prompt, navigated to a subfolder of the project and ran the following commands:

    • Removed the bin and obj folders, then ran "make -j"; it took 35 seconds to complete
    • Removed again bin and obj, then ran "make"; it took 60 seconds
    • Removed again bin and obj, then ran "make -j12"; it took 36 seconds

    That sounded promising, so I tried a similar approach using the parent Makefile, so I navigated to the project root and ran the following commands:

    • Removed the bin and obj folders only of one sub project (the same I tested before), then ran "make -j"; it took 3 minutes and 45 seconds
    • Removed again the bin and obj folders of the sub project, then ran "make"; it took 4 minutes and 5 seconds
    • Removed again the bin and obj folders of the sub project, then ran "make -j12"; it took 4 minutes and 5 seconds

    In few words, after having generated the Makefiles with QMake, if I run "make -j12" from the command line for a single sub-project it works while running the same command for the main project makes the build being processed with a single job.

    Any idea about the reason?


  • Lifetime Qt Champion

    Hi and welcome to devnet,

    Since you are mentioning subproject. are you using the suddirs project template ?



  • @sgaist said in Android Project - Make.exe parallel jobs:

    Hi and welcome to devnet,

    Since you are mentioning subproject. are you using the suddirs project template ?

    Hi @SGaist and thanks for your reply. Yes, I am using the subproject template, why are you asking?


  • Lifetime Qt Champion

    Can you show how you wrote the dependencies ?
    If you use the ordered option, you might be killing the build parallelisation.



  • @sgaist said in Android Project - Make.exe parallel jobs:

    Can you show how you wrote the dependencies ?
    If you use the ordered option, you might be killing the build parallelisation.

    It's exactly as you said:

    TEMPLATE = subdirs
    
    SUBDIRS = \
        proj1 \
        proj2 \
        main \
        test
    
    CONFIG += ordered
    

    I know that by using ordered I'm asking to compile projects sequentially, but I cannot explain why the build system is avoiding the parallelism within the sub-projects.

    Moreover the same template is working perfectly when I use Desktop as target instead of Android...

    Do you have any idea about how I could get rid of this behavior?

    Edit: I use JOM to build the Desktop target.



  • @paolot

    Apparently the make.exe delivered by google with Android toolchain is not observing that flag.

    I had noticed also that apparently the compilation took quite long. Certainly I am using subdir template which explains the sequential compilation of each dynamic lib, but also within a lib the compilation is sequential. That is due to make.exe as from Android SDK. For testing I have ensured that the parameter -j8 is set through additional make argument. This argument is set:

    12:16:08: Starting: "C:\Users\xxx\AppData\Local\Android\Sdk\ndk-bundle\prebuilt\windows-x86_64\bin\make.exe" -j8
    

    In order to allow a parallel compilation through creator an independend tool like jom.exe would be required in that case. It may be downloaded from https://wiki.qt.io/Jom . However, the page states explicitly that this is a replacement for nmake. Therefore, it might not be suitable for cross-compilation for Android.



  • @koahnig said in Android Project - Make.exe parallel jobs:

    @paolot

    Apparently the make.exe delivered by google with Android toolchain is not observing that flag.

    I had noticed also that apparently the compilation took quite long. Certainly I am using subdir template which explains the sequential compilation of each dynamic lib, but also within a lib the compilation is sequential. That is due to make.exe as from Android SDK. For testing I have ensured that the parameter -j8 is set through additional make argument. This argument is set:

    12:16:08: Starting: "C:\Users\xxx\AppData\Local\Android\Sdk\ndk-bundle\prebuilt\windows-x86_64\bin\make.exe" -j8
    

    In order to allow a parallel compilation through creator an independend tool like jom.exe would be required in that case. It may be downloaded from https://wiki.qt.io/Jom . However, the page states explicitly that this is a replacement for nmake. Therefore, it might not be suitable for cross-compilation for Android.

    That is what I thought at the beginning, but I started having doubts when I discovered that make.exe from the Android SDK works concurrently when launched from any sub-project folder. Are we sure there is not anything wrong with the project structure or the root level Makefile?


  • Lifetime Qt Champion

    The ordered keyword is a quick and easy way to build the subprojects in the order you want.

    The correct way is described in the documentation but also on this nice article.



  • @paolot

    Basically you should be able to start the Android make on a command prompt outside of Qt creator at the same location where qt creator does. When it would be concurrent under these conditions it would point to some specifics of the Qt run environment for starting Android make.
    I have not experienced such a behaviour so far. Therefore I consider as odd.



  • @sgaist said in Android Project - Make.exe parallel jobs:

    The ordered keyword is a quick and easy way to build the subprojects in the order you want.

    The correct way is described in the documentation but also on this nice article.

    @SGaist thanks for the article, it is very interesting and useful, anyway even by switching to the suggested configuration (avoiding ORDERED) did not solve the issue for me.

    @koahnig said in Android Project - Make.exe parallel jobs:

    @paolot

    Basically you should be able to start the Android make on a command prompt outside of Qt creator at the same location where qt creator does. When it would be concurrent under these conditions it would point to some specifics of the Qt run environment for starting Android make.
    I have not experienced such a behaviour so far. Therefore I consider as odd.

    I completely agree with you, my only fear is that QMake is doing something wrong when creating the top-level Makefile and it's somehow preventing the build process to go parallel. I could try writing a custom Makefile just building the subfolders, I'll let you know!


Log in to reply