How to increase object file section capacity

[crosier011]

I need to store large amounts of data in a single cpp file but I get the following errors after it reaches a certain size:

mingw32-make[1]: *** [Makefile.Debug:780: debug/Data.o] Error 1

and

mingw32-make: *** [Makefile:45: debug] Error 2

Initially in the "General Messages" tab of the build details window it said something along the lines of "File too big". It doesn't say that anymore, but the errors are exactly the same.

I tried setting QMAKE_CXXFLAGS += -bigobj in my .pro file but it wasn't recognized. I then tried setting
QMAKE_CXXFLAGS += -Wa,-mbig-obj, which were recognized but did not increase the data limit.

[SGaist]

Hi and welcome to devnet,

What kind of data do you need to store in your cpp file ?
And out of curiosity, why ?

[crosier011]

I am storing arrays of objects containing primarily Qstring and integer variables. The cpp file will act as a database for my program.

[SGaist]

Wouldn't it be simpler to have that in a SQLite database ?

[crosier011]

I don't believe so.

[Joel Bodenmann]

Would you care to elaborate why you don't think so?

If you can wrap the data into a C++ file you can certainly wrap it into an SQLite database.
Note that SQLite is exactly that: Lite/Light. The database itself is stored in a file and doesn't require the complicated setups of a dedicated database server etc. that you might know from MySQL, PostgreSQL and similar solutions.
If this is about deployment: Create your SQLite database and just add the database file as a resource to your Qt project. Then it's basically the same workflow as having the data in a C++ file.

Qt provides the necessary high-level SQL classes to simply access your SQLite database.

[crosier011]

All I would like to know is how I can increase object file capacity so as to be able to store more data in the cpp file.

[crosier011]

@Joel-Bodenmann If I started with SQLite then it might be somewhat more efficient for me to program. But I have designed my project around this single file database and would have to rework quite a few things. I don't think its worth it. Ultimately I need to be hand copying a lot of data and neither method can speed that up.

[JonB]

@crosier011
This seems to be a MinGW issue, not Qt. I don't know that anyone here will have an answer. It seems your -Wa,-mbig-obj is the best bet. If that doesn't work you have a problem. It maybe does not help that you are doing a 32-bit rather than 64-bit build.

[OOI: Just how big is this CPP file of yours?]

If I were you I would delete everything and do a fresh, clean build (with the -mbig-obj). Get the exact error message back, in case there is a clue.

Obviously I agree with everybody that maintaining a massive source file for data is not a good approach. The proof of the pudding is that you are having this problem now....

[kshegunov]

Please provide the whole compile line that's executed during the build.

[mzimmers]

@JonB I just encountered this error, as evinced by this message:

mocs_compilation.cpp.obj: too many sections (32884)
C:\Users\MICHAE~1.ZIM\AppData\Local\Temp\cclcUK6a.s: Assembler messages:
C:\Users\MICHAE~1.ZIM\AppData\Local\Temp\cclcUK6a.s: Fatal error: can't write 11 bytes to section .text of appNgaIcdFw/CMakeFiles/appNgaIcdFw.dir/appNgaIcdFw_autogen/mocs_compilation.cpp.obj: 'file too big'

Your suggestion fixes it, but the result is a huge executable (like 3X the size of a standalone static image). So, something is just not right here. Oh - I only get this build error on Windows - the same project does just fine on Ubuntu.

I'm wondering whether there's some more cmake magic that might help ameliorate this...

[Joel Bodenmann]

I encountered a similar problem in a different project. However, in my scenario it wasn't large portions of binary data but just a lot of templated boost code.
When using MinGW, -mbig-obj wasn't enough. I also had to go for an O3 optimization level.
As the same project also needed to compile with MSVC I used CMake generator expressions: https://github.com/Tectu/malloy/blob/9b7145a8834d387ccc25f51f8345fb31f5e17538/lib/malloy/target_setup.cmake#L8

[mzimmers]

@Joel-Bodenmann I have absolutely no idea why my image is getting so large. It's not a huge code base at all, and I'm not importing anything large. I wonder if using a different toolchain might help...?

[Joel Bodenmann]

If you're importing an image, why not just use Qt's RCC?

[mzimmers]

@Joel-Bodenmann sorry - I used the term "image" to refer to the built executable. (That's what we used to call them back in the old days.) I'm not importing anything that should be contributing to this issue.

[IgKh]

@mzimmers said in How to increase object file section capacity:

mocs_compilation.cpp.obj

Looks like you got bit by AUTOMOC merging all meta objects into a single cpp file. Static QMetaObject are full of strings, and you probably have a lot of QObjects. See https://gitlab.kitware.com/cmake/cmake/-/issues/22161

[mzimmers]

@IgKh Ouch. I'd guess I have on the order of ~100 subclasses of QObject. Hard to believe that would break the build, but I guess it does.

So, regarding the solution you posted, is this something that can be done incrementally? In other words, if I just add the MOC include in a few files, will that show a minor improvement, or do I need to do it to everything in order to get results?

Thanks...

[IgKh]

@mzimmers yes, you can do this incrementaly

[Christian Ehrlicher]

You should always include the moc_foo.cpp in your source file instead relying on automoc's foo_compilation.cpp magic. Then you will also get more diagnostic output (at least for gcc) since the compiler sees the whole class in one compilation unit.

[mzimmers]

@Christian-Ehrlicher OK, and is this the preferred syntax (taken from the solution @IgKh provided)?

// myobject.cpp
#include "myobject.h"

MyObject::MyObject(QObject *parent)
    : QObject(parent)
{
}

#include <moc_myobject.cpp>