Speed up make or/and ninja building proccess

JacobNovitsky

Is there any way to ignore some unnecessary lines (pre-compiled headers way) to speed up make and ninja building process?

sierdzio

Things to look into:

always pick ninja, make is always slower
use jumbo builds (UNITY_BUILD, UNITY_BUILD_MODE, UNITY_BUILD_BATCH_SIZE)
use ccache
clean up your includes, use forward declarations as much as possible, never include whole Qt modules

JoeCFD

@JacobNovitsky Try meson out. It seems that meson is (much) quicker than cmake. I tested to install a third party package with cmake and meson. Meson is way faster than cmake. I am planning to switch to meson soon.

JacobNovitsky

@sierdzio hi! thank you for the reply!

always pick ninja, make is always slower -> is there something I shoud change in Qt Creator preferences?

sierdzio

@JacobNovitsky said in Speed up make or/and ninja building proccess:

@sierdzio hi! thank you for the reply!

always pick ninja, make is always slower -> is there something I shoud change in Qt Creator preferences?

No, Qt Creator uses ninja by default.

SimonSchroeder

@sierdzio said in Speed up make or/and ninja building proccess:

never include whole Qt modules

Well, with pre-compiled headers (for the Qt modules) this should not be much of a problem. (Though, I don't advise to include whole Qt modules.) Most important is that you only have 3rd party libraries in your pre-compiled header and not your own code. Otherwise a change in one of your headers will trigger a recompile of the pre-compiled header and as a dependency a recompile of everything else as well.

I have tried several different tools for caching on Windows. This was way too complicated to achieve for a qmake-based project. In the end I decided on FASTBuild which is a single tool for unity/jumbo builds, caching and distributed compilation. However, it is not compatible with CMake, but replaces CMake as well (though, probably not the best replacement for CMake). I have created a bunch of scripts to convert a qmake-based project into a FASTBuild project (https://github.com/SimonSchroeder/QMake2Fastbuild). I would love for CMake to have a FASTBuild generator.

JacobNovitsky

@SimonSchroeder, Simon, thanks for sharing, I just about to research whole thing and make something like your script, maybe using some cache methods to discard unnecessary transactions

Ubuntu, 6.2.4, gcc, qmake
already have some automation script for qmake, but never dived into building engine yet
how to amend your project for ubuntu?

SimonSchroeder

@JacobNovitsky said in Speed up make or/and ninja building proccess:

how to amend your project for ubuntu?

Most of the things should work right away on Linux as well. The batch scripts (in the scripts folder) mostly work as a bash script also. grep and sed should be available by default on Linux or are easy to install through the package manager. Only generateInputFiles4Fbuild.bat needs a little more adjustment. You don't need to load Visual Studio (or any compiler for that matter) and just need to adjust the path to qmake. Also, rmdir needs to be replace by a simple rm -r.

A little bit more involved is the adjustment of the .bff files. It should be fairly easy to adjust the paths of the .bff includes in the Qt subfolder. Certainly you need to replace the Visual Studio folder by a GCC or Clang folder. My inspiration for the general setup came from the .bff files for building FASTBuild itself. Have a look over there how to set up a GCC or Clang folder. The only additional thing they do is to copy the compiler explicitly to other machines during distributed builds. The toolchain (i.e. Windows or Linux, GCC or Clang (if you have both as option)) is selected in the fbuild.bff file. IIRC you can have a look at the FASTBuild setup itself for how to select the toolchain inside this file.

One last thing is which compiler/linker flags to choose. For this I ran qmake once initially and had a look at the generated Makefile. These are the flags I set inside fbuild.bff and I think there are more (especially some default libs) inside the Visual Studio folder.