How does #include "moc_*.cpp" work?
-
@KH-219Design said in Reduce compilation time:
I am intrigued. Does anyone have an explanation for why/how this works?
Normally, every *.cpp file is compiled separately into their own *.o file. However, if you #include one *.cpp file within another, then they are merged into a single *.o file so that reduces the number of compilation units. Furthermore, when the source files are merged, the compiler can apply optimizations that it can't do with separate files.
Also, how/why did anyone even think of trying it? (The thought has never occurred to me.)
This technique is not unique to Qt. The SQLite project takes this to the extreme: It merges all of its source files into a single unit! https://www.sqlite.org/amalgamation.html
Having said that, the Qt engineers are a fan of this technique: https://lists.qt-project.org/pipermail/development/2020-May/039654.html
I don't bother doing it for my small projects, however.
-
@JKSH This has been a very eye-opening conversation. Thank you!
I checked the online docs to see if I had just never fully read about proper
moc
usage, but all I found was:This generated source file is either #include'd into the class's source file or, more usually, compiled and linked with the class's implementation.
Here: https://doc.qt.io/qt-5/metaobjects.html
Meanwhile, based on the information you have shared on this thread, I now know far more:
- KDE code and internal Qt code prefers the tactic of "#include" the moc_file.cpp in the corresponding cpp.
- This results in fewer
*o
files fed into the linker. - This has been shown to speed up builds.
- This can at times enable the reduction of the header-to-header include graph, per this email, which is a tactic I had mentioned on this other forum thread as being my main weapon overall against long rebuild times (in any C/C++ project, Qt or not).
- Per the famous Thiago Macieira (see here), the
#include
-your-moc will also allow Clang to provide better diagnostics regarding your class.
This builds an extremely compelling case for using this tactic.
-
@JKSH said in Reduce compilation time:
Normally, every *.cpp file is compiled separately into their own *.o file. However, if you #include one *.cpp file within another, then they are merged into a single *.o file
That much makes sense.
But when using qmake, I always assumed that some combination of
qmake
plusmoc
plus some other Qt-specific tooling was both generating a moc_file.cpp on my behalf and also appending that generatedcpp
to the build commands and then adding the correspondingo
orobj
to the linker command. So I would think that if I manually add a pound-include, I am just adding redundant work and not lessening the overall work of the build. -
@KH-219Design said in Reduce compilation time:
But when using qmake, I always assumed that some combination of
qmake
plusmoc
plus some other Qt-specific tooling was both generating a moc_file.cpp on my behalfThat's right.
and also appending that generated
cpp
to the build commands and then adding the correspondingo
orobj
to the linker command.No. The linker is too late -- it only runs after all the compilation is already complete.
The tools cannot combine multiple *.cpp files into a single translation unit. (Well, technically they can enable Single Compilation Unit to force the merge, but that that's not a good solution because it disables parallel compilation so your overall build time will become even longer: https://en.wikipedia.org/wiki/Single_Compilation_Unit)
So I would think that if I manually add a pound-include, I am just adding redundant work and not lessening the overall work of the build.
If you manually add a pound-include, you'll be doing work that the automated tools can't do.
The only way to perform the merge is if you write the
#include
into your *.cpp file. The Qt tools won't modify your source code. -
@JKSH I think there must be something about your build configuration that you might take for granted that I am still completely in ignorance of.
There are now at least 3 experienced Qt developers participating in this thread that all seem to agree that pound-including "moc_myfile.cpp" is an accepted tactic. It is clear to me that this must be true. But I still cannot figure out how it is being done.
Somewhere between
qmake
taking my*pro
file as input and before I get my final app binary, somewhere in there some Qt tool is definitely addingmoc_myfile.o
to a linker-related command. Concretely, I see this in my build log:g++-7 -Wl,-rpath,'$ORIGIN' -Wl,-rpath,'$ORIGIN' -Wl .... -o libadmin_app.so.1.0.0 navigation.o pending_configuration.o pool_of_products.o resources_admin.o cli_options.o lib.o logging_tags.o resources.o qrc_libresources.o moc_navigation.o moc_pending_configuration.o moc_pool_of_products.o moc_logging_tags.o ....
... and nowhere in any of my own files do I ever use the token "moc" for any filename or include directive or line of any
*pro
file.So the fact that I see
moc_navigation.o moc_pending_configuration.o moc_pool_of_products.o moc_logging_tags.o
appearing as linker inputs in my build log is why I made my statement about the Qt toolchain doing all 3 of : generating moc_x.cpp, compiling moc_x.cpp to get moc_x.o, then adding moc_x.o to the arguments fed to the linker.So because I continue to be very curious about this idea of explicitly pound-including my own moc(s), I added this line to my code:
#include "moc_navigation.cpp"
And that causes my build to fail with many errors such as:
build/src/lib_app/moc_navigation.cpp:101: multiple definition of `Navigation::staticMetaObject' build/src/lib_app/moc_navigation.cpp:158: multiple definition of `Navigation::metaObject() const' build/src/lib_app/moc_navigation.cpp:163: multiple definition of `Navigation::qt_metacast(char const*)'
Which is tending to reinforce my original thinking that by adding
#include "moc_navigation.cpp"
to my own code manually, I duplicated work that Qt was doing (resulting in duplicate symbols).And now I feel a bit rude for arguably hijacking someone else's thread. I'll happily step aside if asked to.
OTOH, this is still in service to the greater good of the topic of Reducing Compilation Times and of the (still fascinating and mysterious to me) idea of putting #include "moc_file.cpp" inside the "file.cpp"
-
Correction (in part): I did a full clean and rebuild, and now I no longer see any
multiple definition
errors and the whole build succeeds.But part of my question still stands if anyone else is interested: which part of the Qt/qmake toolchain was the "smart" piece that became aware of my
#include "moc_x.cpp"
line and then intelligently chose NOT to do what it was previously doing (which was the previous 3-step automated dance of generating the cpp, then theo
, then telling the linker abouto
)? -
@KH-219Design said in How does #include "moc_*.cpp" work?:
But part of my question still stands if anyone else is interested: which part of the Qt/qmake toolchain was the "smart" piece that became aware of my
#include "moc_x.cpp"
line and then intelligently chose NOT to do what it was previously doing (which was the previous 3-step automated dance of generating the cpp, then theo
, then telling the linker abouto
)?qmake is responsible for this. It scans your code and your *.pro file to generate a Makefile. Your compiler and linker follows the "recipe" contained in the Makefile. When you don't #include "moc_myclass.cpp", then your QObject-based class spans 2 separate *.o files:
SOURCES = ../src/main.cpp \ ../src/myclass.cpp moc_myclass.cpp OBJECTS = main.o \ myclass.o \ moc_myclass.o
When you #include your moc file, qmake sees this and adjusts the Makefile accordingly, so your class is fully contained within 1 *.o file:
SOURCES = ../src/main.cpp \ ../src/myclass.cpp OBJECTS = main.o \ myclass.o
And now I feel a bit rude for arguably hijacking someone else's thread. I'll happily step aside if asked to.
Thank you for being considerate. This discussion is now forked into its own thread.
-
@JKSH This has been a very eye-opening conversation. Thank you!
I checked the online docs to see if I had just never fully read about proper
moc
usage, but all I found was:This generated source file is either #include'd into the class's source file or, more usually, compiled and linked with the class's implementation.
Here: https://doc.qt.io/qt-5/metaobjects.html
Meanwhile, based on the information you have shared on this thread, I now know far more:
- KDE code and internal Qt code prefers the tactic of "#include" the moc_file.cpp in the corresponding cpp.
- This results in fewer
*o
files fed into the linker. - This has been shown to speed up builds.
- This can at times enable the reduction of the header-to-header include graph, per this email, which is a tactic I had mentioned on this other forum thread as being my main weapon overall against long rebuild times (in any C/C++ project, Qt or not).
- Per the famous Thiago Macieira (see here), the
#include
-your-moc will also allow Clang to provide better diagnostics regarding your class.
This builds an extremely compelling case for using this tactic.
-
This qualifies as SOLVED in my book. Is there something I can click to mark it as such?
-
@KH-219Design said in How does #include "moc_*.cpp" work?:
This qualifies as SOLVED in my book. Is there something I can click to mark it as such?
I think only @JKSH can make it solved, as the "creator" of this topic ;) I'm sure he will do so when reads this the next time
-
@J-Hilk said in How does #include "moc_*.cpp" work?:
@KH-219Design said in How does #include "moc_*.cpp" work?:
This qualifies as SOLVED in my book. Is there something I can click to mark it as such?
I think only @JKSH can make it solved, as the "creator" of this topic ;) I'm sure he will do so when reads this the next time
I marked the answer. @JKSH if you disagree feel free to change it or ping me.
@KH-219Design @JKSH many thanks, I've learned a lot here, too!