How to improve qtcreator parsing performance when using a recurring system call in qmake files

devjb

I am pimping my *.pro files for better flexibility during automated builds. Therefore, I want to be able to inject a build date and time via the qmake call. However, I also want to have defaults defined in the *.pro file, which are used, when no date and time have been passed on qmakes invokation.

Basically the entries for this look like so:

# builddate.pri

win32{
	PRODUCT_BUILDDATE_COMBINED=$$system(powershell -Command Get-Date -Format "yyyy.MM.dd.HH.mm")
}
unix{
	PRODUCT_BUILDDATE_COMBINED=$$system(date +%Y.%M.%d.%H.%m)
}

!defined(PRODUCT_BUILDDATE_YEAR,var){
	PRODUCT_BUILDDATE_YEAR=$$section(PRODUCT_BUILDDATE_COMBINED, ., 0, 0)
	message("Build date part PRODUCT_BUILDDATE_YEAR has not been given, got it from the current system: $${PRODUCT_BUILDDATE_YEAR}")
}

However, since I have a subdirs project with a lot of subprojects, where I everywhere include the above *.pri file, as soon as the system call is included, the qtcreator project tree parsing slows down dramatically. It needs about one minute to finish updating the project tree, where without the system call it takes a few seconds.

I would love to have a possibility, where I do only have to make the system call once I need it for the first time and store the result in a variable, which I then can get in any other sub project. However I have no idea, how this could be achieved, since no concept of variable sharing or propagation in a recursive qmake subdir run exists.

Maybe someone has an alternative idea how to achieve the goal?

Thanks in advance

JonB

@devjb
I don't use any of this so I have no idea whether it would work. But can these makefiles include other files, with makefile statements in them? Then maybe: at the top level it sends that PRODUCT_BUILDDATE_COMBINED=... statement, with date part filled in from the system call, to a file, which is included into your other files?

Bonnie

I've read this post which is also about calling system command. And it mentioned QMAKE_SUBSTITUTES.
It reminds me that I've also read this wiki page about using QMAKE_SUBSTITUTES to create .qmake.cache in the top-level build folder. (Do note what I'm saying is the Qt 4 based one, it is still valid. And the other Qt 5 based part is not suitable for your case since we need to use QMAKE_SUBSTITUTES.)
So I think maybe they can be combined. Actually what I tried is quite similar to what @JonB said.

This is tested by a simple subdir project, I'm not sure if it can work with your "a lot of subprojects".
And qmake must be called to update the time.

In the top-level subdirs .pro file:

TEMPLATE = subdirs

SUBDIRS += \
    subdir1\
    subdir2\
    ....

win32{
	TEMP_VAR_NAME=$$system(powershell -Command Get-Date -Format "yyyy.MM.dd.HH.mm")
}
unix{
	TEMP_VAR_NAME=$$system(date +%Y.%M.%d.%H.%m)
}
QMAKE_SUBSTITUTES += .qmake.cache.in

Create a .qmake.cache.in file with the top-level subdirs .pro file, containing following content (As I tested it must NOT contain any BOM.)

PRODUCT_BUILDDATE_COMBINED=$$TEMP_VAR_NAME

This will create a .qmake.cache in the top-level build folder and it can automatically be read by the subprojects.
Don't forget to remove the system call part from your .pri.

devjb

@Bonnie Thank you very much for pointing this out.

I indeed was able to successfully set it up like described. Actually, I additionally found an even simpler approach to cache a variable.
You can simly use cache(MYVARIABLE, set), to cache the content of a variable.

While caching works now for regular qmake calls, it does not change the way how Qtcreator is parsing the project tree, no matter which of the both approaches I tried.
I think the project tree parsing is not respecting the cache instructions.

~~This is really sad, because that is the way bigger problem, since it happens all the time, wehnever I edit a file.~~

~~So I would call this half solved. I keep it unsolved in the forum for now, maybe someone has an idea how to get caching also for Qtcreator project tree parsing.~~

Edit: after one full project tree parsing after startup, it seems to work also for the project tree. It seems that it needs a full parse on each fresh project opening.

I also noticed, that the slowdown in a real qmake run when nothing is cached is not that much amount as in the project tree parsing.

Bonnie

@devjb
That's good to know. :)

devjb

It looks like the root cause of the issue has been fixed in the backend. system() calls during parsing are apparently now an opt-out:

https://bugreports.qt.io/browse/QTCREATORBUG-24551
https://codereview.qt-project.org/c/qt-creator/qt-creator/+/317710