Linkage and initialisation order of static variable (issue on Linux env only)

mbruel

Hi,
I'm getting a project where some classes have statics and initialize them straight in their cpp (without an init_static function).
I've a class P with a static variable sVarP that is initialized using a static function P::create that uses in parameter the static variable sVarE of another class E.
The static variable E::sVarE is also initialized with a call of a static function E::create.
Something like this:

P.cpp:
Elem1 * P::sVarP = P::create(true, E::sVarE);
Elem1 will store the pointer on E::sVarE


E.cpp
Elem2 * E::sVarE = E::create(1, "plop", false);

I've an linkage issue and/or a problem of initialisation order of the statics on Linux. Basically P::sVarP is initialized before E::sVarE so I end up with P::sVarP holding a pointer to NULL.

Is there a way to play on the order of those static initialisations?
I've read that we can if we use static libraries by telling the compiler the order to use them but I'm wondering if it could be possible directly with the object files .o.
Plus I would like to still be available to debug with QtCreator and I'm not sure if this option will allow it and how to do it if not manually using the ar command to generate static libraries of all the object files and then link them in the good order to create the final executable.

Any suggestions?

PS: I don't this initialisation issue on Windows but I'd like to migrate the project on linux and now I'm getting a segmentation fault before arriving to main because of the order of the initialisation of my static. During the following stack, P is initialized before E...

Thread 1 (Thread 0x7ffff7fb9840 (LWP 12503)):
#0  __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at ../../P.cpp:896
No locals.
#1  0x000000000051658f in _GLOBAL__sub_I_P.cpp(void) () at ../../P.cpp:1727
No locals.
#2  0x00000000007cf4bd in __libc_csu_init ()
No symbol table info available.
#3  0x00007ffff1d61ad5 in __libc_start_main (main=0x437084 <main(int, char**)>, argc=1, argv=0x7fffffffe838, init=0x7cf470 <__libc_csu_init>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe828) at libc-start.c:246
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {0, 140737351949767, 140733193388033, 0, 140737333576216, 140737250656680, 0, 0}, mask_was_saved = 4412096}}, priv = {pad = {0x7ffff7df02e5 <_dl_runtime_resolve+53>, 0x1c, 0x7cf470 <__libc_csu_init>, 0x7fffffffe838}, data = {prev = 0x7ffff7df02e5 <_dl_runtime_resolve+53>, cleanup = 0x1c, canceltype = 8189040}}}
        not_first_call = <optimized out>
#4  0x00000000004352e9 in _start ()
No symbol table info available.

kshegunov

@mbruel said:

I've an linkage issue and/or a problem of initialisation order of the statics on Linux. Basically P::sVarP is initialized before E::sVarE so I end up with P::sVarP holding a pointer to NULL.

Yeah, that's what happens when you write in C, instead of C++.

Is there a way to play on the order of those static initialisations?

Nope, and you shouldn't depend on the order anyway.

I've read that we can if we use static libraries by telling the compiler the order to use them but I'm wondering if it could be possible directly with the object files .o.

Nope. The order of static libraries bears no consequence, as everything that's an archive (*.a) will be linked into the same final binary with the proper reordering.

Plus I would like to still be available to debug with QtCreator and I'm not sure if this option will allow it

Yep. Creator will not flinch and will not prevent you from debugging static libraries.

Any suggestions?

Don't use static variables, singletons and other such nonsense. Write a good object-oriented code, in the end C++ is supposed to be object oriented.

I don't this initialisation issue on Windows

Because the order of initialization is an implementation detail and will vary from compiler to compiler, from OS to OS.

During the following stack, P is initialized before E...

May I ask, why is it so important that those are static?

Kind regards.

mbruel

@kshegunov said:

Yeah, that's what happens when you write in C, instead of C++.

well it is pure C++ code with well defined classes that allows to instantiate objects but also to use static members to share data (constant or not) between all of them and/or with the outside world.
I've to admit, it's the first time I see such "complex" code where the statics are initialized with a static method (from the same class) that can use arguments that are other statics from another class (that would need also a function call to be initialized).

I'd normally have a init_static static function in all my classes that needs to initialize some static members and in the main, first thing I'll do would be to call an init_static function that would call all the one of my classes in the order I want. I guess that would be the proper way to go.

But I'm just getting the hand over on a project that is coded like this: initializing the static variable straight in the cpp of the classes using function calls...
It was done under windows and I'm not sure why but they never encounter any linkage issue like I've on Linux.
In a way I can understand why they do it if they are not aware of the problem I'm having. Cause instead of initializing the variable to NULL and then having to affect them to the proper value in an init_static function, you're writing half lines of code. Plus you're kind of sure that the main program won't forget to call this init_static method... And also, not all the statics of a class will use a static method call so some are directly initialized in the .o and won't be affected at run time during the init... All of your statics are initialized at the same place.

Having say all this and being aware of the problem I understand this is not the way to go and I'm seriously thinking about creating those init_static funtion.
The problem is that the software is linking ok on Windows and I'm the one who prefer to work on a Linux environment although it is not really needed for the client...
I'm having around 1500 classes. Not all of them use statics, but potentially a few... I didn't start the handover yet, just got access to a snapshot of the code so I can check if I can compile it, run it, and check a bit how it is done. So I don't know yet how it works but I guess it would be my pain to understand the order of the static initialisation... People told me already that I should rather just use windows... I've a VM for now, but I'd really rather stick on my debian!

May I ask, why is it so important that those are static?

I didn't look in details yet, but basically I think it is mainly on Factory classes where some public types are defined. I suppose they are constants, used for category checking and/or to create clones.

Have a good night :)

PS: thanks for your replies kshegunov ;)

kshegunov

@mbruel

well it is pure C++ code with well defined classes that allows to instantiate objects but also to use static members to share data (constant or not) between all of them and/or with the outside world.

That's what I meant. The moment you start sharing data between all objects is the moment you throw OOP away and start writing procedural all together. It basically means all the objects are coupled (through the statics) and depend on the globals, and to pile up when the globals depend on one another, well then everything goes to hell.

I've to admit, it's the first time I see such "complex" code where the statics are initialized with a static method (from the same class) that can use arguments that are other statics from another class (that would need also a function call to be initialized).

Happens more often than not, sadly.

I'd normally have a init_static static function in all my classes that needs to initialize some static members and in the main, first thing I'll do would be to call an init_static function that would call all the one of my classes in the order I want. I guess that would be the proper way to go.

You can do that before main, but there are some conditions. Consider this example:

class Global1  {}
class Global2
{
    Global2(Global1 *); //< Requires Global1 to be initialized first
}

Global1 * globalVar1 = NULL; //< This is always run before non-trivial initializations
Global2 * globalVar2 = NULL;

static class __initStatics
{
    __initStatics()
    {
        // This is run before main()
        globalVar1 = new Global1(); //< Create the first global
        globalVar2 = new Global2(globalVar1); //< Okay the first one has been initialized
    }

    ~__initStatics()
    {
        // This is run after main(), delete the objects in reverse (basically what the stack'd have done)
        delete globalVar2;
        delete globalVar1;
    }
} __staticInitializer;

So depending on your globals and how they are initialized, you can have one such static object that initializes all other global variables (WARNING: they have to be created in the heap!). If you have non-trivial initialization and the globals are created in the stack, there is no definitive way to ensure proper order of initialization. For example:

Global1 globalVar1;
Global2 globalVar2(&globalVar1); //< Global2's constructor might be called before Global1's constructor has run. Extra bad!

It was done under windows and I'm not sure why but they never encounter any linkage issue like I've on Linux.

This is unrelated to linkage itself, this is the loader. The loader maps the binaries into the (virtual) memory and executes the entry point of the application. So while there are (very) few well defined rules on how globals are initialized (trivial initializations are always run before non-trivial ones) there's no specific, or rather defined, order inside each of those init steps.

In a way I can understand why they do it if they are not aware of the problem I'm having. Cause instead of initializing the variable to NULL and then having to affect them to the proper value in an init_static function, you're writing half lines of code. Plus you're kind of sure that the main program won't forget to call this init_static method...

It may sound harsh or even insulting, but this is (for 99% of cases) a bad style and lack of understanding of OOP and its principles.

The problem is that the software is linking ok on Windows and I'm the one who prefer to work on a Linux environment although it is not really needed for the client...

It will still link fine on Linux. It just might not run.

People told me already that I should rather just use windows...

Another bad advice, forgive me for saying. What happens when/if the windows' loader's implementation changes and suddenly all this code stops working ...? You'd be again in the same predicament ...

but I'd really rather stick on my debian!

Amen! :)

I didn't look in details yet, but basically I think it is mainly on Factory classes where some public types are defined.

Another overly exploited design pattern (or in most cases anti-pattern) that Java devs love. In my (about) 12 years of programming I had to use factory for a handfull of cases (usually with serialization/deserialization).

Good luck.
Kind regards.

alex_malyu

There is nothing wrong with singleton pattern and nothing wrong with static (class members, functions, and variables ) when used properly. But proper usage for static variables is very limited.

If you need static variable of no built in type mostly likely you better use singleton pattern (and in this case you have no any problem with initialization order cause you never access this variable directly,
but through a function, which would allocate required resource when necessary, and this means you never have problems with initialization order - kind of lazy initialization happens.

In most other cases there are much better places where you can instantiate these objects.
You do not need init_static functions. In case of Qt one of such places would be class derived from QApplication (QCoreApplication ) or your main widget class.

kshegunov

@alex_malyu

But proper usage for static variables is very limited.

Indeed my point.

and in this case you have no any problem with initialization order cause you never access this variable directly,
but through a function, which would allocate required resource when necessary, and this means you never have problems with initialization order - kind of lazy initialization happens

Firstly, this isn't the case here. And secondly it generates yet another battery of problems.

Who cleans up the memory?
Usually the answer is No one, we leave that dirty work to the OS, 'cause that's just it ...
What happens when we call MyNiceSingleton::instance() from two different threads?
Oops, I didn't mean to create two objects and leak one of them and/or corrupt the pointer that references it
And we go about solving a needless multithreading issue for creation of the object alone.

In case of Qt one of such places would be class derived from QApplication (QCoreApplication ) or your main widget class.

Why??! Why do you need to derive from QApplication??! Have a nice separate object that's independent of QApplication sitting comfortably in the main() stack frame, and do what QApplication does ...

class MyGlobalObject
{
public:
    MyGlobalObject()
    {
        Q_ASSERT(!ref);  //< Qt does this check with an if construct and logs the error
        ref = this;
    }

    static MyGlobalObject * instance()
    {
        return ref;
    }

private:
    static MyGlobalObject * ref;
}

MyGlobalObject * MyGlobalObject::ref = NULL; //< This goes in the .cpp obviously

So you end up using this like this:

int main(int argc, char ** argv)
{
    QApplication app(argc, argv);
    MyGlobalObject ohWowItsASingletonObject;  //< And after this MyGlobalObject::instance() will return the initialized object
}

mbruel

@kshegunov said:

You can do that before main, but there are some conditions. Consider this example:

class Global1  {}
class Global2
{
    Global2(Global1 *); //< Requires Global1 to be initialized first
}

Global1 * globalVar1 = NULL; //< This is always run before non-trivial initializations
Global2 * globalVar2 = NULL;

static class __initStatics
{
    __initStatics()
    {
        // This is run before main()
        globalVar1 = new Global1(); //< Create the first global
        globalVar2 = new Global2(globalVar1); //< Okay the first one has been initialized
    }

    ~__initStatics()
    {
        // This is run after main(), delete the objects in reverse (basically what the stack'd have done)
        delete globalVar2;
        delete globalVar1;
    }
} __staticInitializer;

That's a nice trick. So if I understand correctly you'd define this static variable in the cpp file of the main outside the main function right? This goes also against the OOP concept ;) haha
What would be the real benefit in term of application loading time compare to simply call an initStatics function in the main that would do the same?
Would it change anything if I was defining the class __initStatics in it's own header and cpp file, compile it in an object .o that would be linked in the executable and just declare a static variable like I said before in the cpp file of my main. I guess no...

This is unrelated to linkage itself, this is the loader.

True I'm mixing linking and loading... I was hoping the linking order could affect the loading but I know now that it is not the case...

It may sound harsh or even insulting, but this is (for 99% of cases) a bad style and lack of understanding of OOP and its principles.

Yes but we could argue that one of the beauty/power of C++ is that it is not limited to a strict object oriented language to offer more flexibility and possibilities to the developer.

Another bad advice, forgive me for saying. What happens when/if the windows' loader's implementation changes and suddenly all this code stops working ...? You'd be again in the same predicament ...

Yeah true but their software is working since several years on Windows and it will have a cost to correct it for potential future changes that could break it...

Another overly exploited design pattern (or in most cases anti-pattern) that Java devs love. In my (about) 12 years of programming I had to use factory for a handfull of cases (usually with serialization/deserialization).

Well I find the factory pattern pretty useful. I see it in fact as an upgrade of a Vector that will hold and often own all the element (in the heap) of a certain type that are used in the system. Plus like in my case holding some special instances (static variable) like in an enum to be models for comparison or cloning.

mbruel

@alex_malyu said:

There is nothing wrong with singleton pattern and nothing wrong with static (class members, functions, and variables ) when used properly. But proper usage for static variables is very limited.

I agree

If you need static variable of no built in type mostly likely you better use singleton pattern (and in this case you have no any problem with initialization order cause you never access this variable directly,
but through a function, which would allocate required resource when necessary, and this means you never have problems with initialization order - kind of lazy initialization happens.

This would be so much overhead to create a singleton class for all my statics just to avoid wrong order in initialisation...

In most other cases there are much better places where you can instantiate these objects.
You do not need init_static functions. In case of Qt one of such places would be class derived from QApplication (QCoreApplication ) or your main widget class.

Well I'm not sure where you're thinking to do it. For me statics are the first thing to initialise, so first place in main or before. I need to have them set up and load my whole model prior to start the GUI.
I agree with kshegunov solution above to use an independant object or even a static function.

kshegunov

@mbruel
Hello,

That's a nice trick. ... This goes also against the OOP concept ;) haha

That's one of the reasons I rarely use it. I had in the past used such approach in a dynamic library (.so/.dll) so I could run initialization/cleanup in a platform independent way for a global resource (API to a driver). For most cases the ordinary advice is still valid: don't use global variables, unless you actually need to (which it seems to me, you don't).

So if I understand correctly you'd define this static variable in the cpp file of the main outside the main function right?

It can be declared and defined in any source file (it doesn't really need a header). The only thing it may need a header for is for the globals it initializes. So for the above example (if put in the cpp) the header would contain only something along the lines of:

// Forward declarations
class Global1;
class Global2;

 // These are in some unspecified translation unit
extern Global1 * globalVar1;
extern Global2 * globalVar2;

What would be the real benefit in term of application loading time compare to simply call an initStatics function in the main that would do the same?

None. The only benefit, if you can call it that, is it'd be run before main() and that it ensures proper cleanup (for when someone might call ::exit() or when the runtime terminates the process). Plus it somewhat encapsulate the ugliness and interdependence of the static initialization, so it can change at some point in the future without affecting other parts of the program.

Would it change anything if I was defining the class __initStatics in it's own header and cpp file, compile it in an object .o that would be linked in the executable and just declare a static variable like I said before in the cpp file of my main. I guess no...

Nope. I've written it in the shortest way possible, but it's still a regular class with a regular (static) object.

Yes but we could argue that one of the beauty/power of C++ is that it is not limited to a strict object oriented language to offer more flexibility and possibilities to the developer.

Sure, but then we go to my initial sentence of my first post. Nice thing about C++ is that it's OO, if you throw that away you're left with all it's quirkiness: (sometimes) awkward syntax/semantics and tons of platform specific details + everything that's inherited from C. So the question would be, is it worth it then?

Yeah true but their software is working since several years on Windows and it will have a cost to correct it for potential future changes that could break it...

I suppose, my point was that it should never had been dependent on the loader initialization order in the first place.

Well I find the factory pattern pretty useful.

I do as well, in (as mentioned) few circumstances. The problem with patterns is they are used whenever someone feels like it, not when actually needed.

I see it in fact as an upgrade of a Vector that will hold and often own all the element (in the heap) of a certain type that are used in the system.

Thus introducing a global state to the application. If you think about it, it is possible to put everything as global variables, no need to bother with members at all ... but then, why complicate your life?

Plus like in my case holding some special instances (static variable) like in an enum to be models for comparison or cloning.

I don't understand this. Aren't virtual methods enough for that?

class Base 
{
public:
    virtual Base * clone() const = 0; //< Here you go, you now have a virtual copy constructor
    virtual bool operator == (const Base &) const = 0; // Tada! ... and you have a virtual comparison operator ...
}

I need to have them set up and load my whole model prior to start the GUI.

Then I suggest you do that in main().

Kind regards.

alex_malyu

@kshegunov said:

Who cleans up the memory?
Usually the answer is No one, we leave that dirty work to the OS, 'cause that's just it ...

Wrong, this is the case when you replace raw pointer with smart pointer as your static object.
So everything is properly deleted.
And this approach is much cleaner than one which require special functions to be called (for allocation and cleaning), which in 99% cases will not be called in case of exception for example.
Static functions for such purpose is a pain to maintain, it is possible that initialization and cleaning function function are called multiple times or especially cleaning function may not be called at all.
Singleton pattern was not invented for fun.

What happens when we call MyNiceSingleton::instance() from two different threads?

You have many more problems related to thread synchronization in such case.
At the same time the problem you mentioned is easy to solve using thread synchronization without actually slowing down the code with pseudo-code below. I will use raw pointer and critical section for simplicity, which should be replaced with smart pointer and mutex for Qt.


MySingleton * getInstance()
{
      if ( pInstance == NULL)
      {
            EnterCriticalSection();
            if (pInstance == NULL)
            {
                  pInstance = new MySingleton();
            }
            LeaveCriticalSection();
      }
      return pInstance;
}

Why??! Why do you need to derive from QApplication??! Have a nice separate object that's independent of QApplication sitting comfortably in the main() stack frame, and do what QApplication does

If you prefer eager approach over lazy you better keep and initialize the data in the object which can guarantee proper allocation and deletion avoiding multi-threading synchronization.
And if you think about it, this would be QCoreApplication subclass in Qt.
On top most of Qt classes must not be instantiated before QApplication. So I see a clear advantage to do such operations within the class which is already a natural singleton and can guarantee a proper allocation and cleaning.

kshegunov

@alex_malyu

Wrong, this is the case when you replace raw pointer with smart pointer as your static object.

It is possible. Still it does look like ever more solutions are needed to solve various problems only because of the choice to use lazy-initialized singleton. I for one dislike that.

And this approach is much cleaner than one which require special functions to be called (for allocation and cleaning), which in 99% cases will not be called in case of exception for example.

When in main you can capture whatever exceptions you like. Also everything in the main() stack is freed whatever the reason for exiting (except for a call to ::exit). Still my previos example can be used to ensure calling of init/deinit routines without that side effects, but as you pointed out it's not lazy-init.

Singleton pattern was not invented for fun.

It wasn't even properly invented ... but I agree it ain't no fun at all ... (teasing of course) :)
Back in C days there were functions written returning properly initialized pointers to global structs, and they weren't even members.

You have many more problems related to thread synchronization in such case.

Yes, there are problems with making the object thread-safe. We are talking here of making the pointer to the object thread-safe!

At the same time the problem you mentioned is easy to solve using thread synchronization without actually slowing down the code with pseudo-code below.

Yes. I'm aware of the double-checked locking paradigm. It may or may not work, however. Look here:
http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html

And if you think about it, this would be QCoreApplication subclass in Qt.

I prefer not to subclass things without a good reason. Initializing a variable, doesn't sound good enough to me, so I will always prefer to keep track of my own junk, and clean it up in the end. Plus, I'm not really changing how the application works in the first place. You're free of course to code however you like, that's just my 2 cents.

Kind regards.

alex_malyu

@kshegunov said:

Yes. I'm aware of the double-checked locking paradigm. It may or may not work, however. Look here:
http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html

Why we would discuss memory models in Java here?
If you are really writing code which should work on any hardware use fully compliant C++11 compiler:

MySingleton& MySingletonSingleton::getInstance() {
static MySingleton instance;
return instance;
}

kshegunov

@alex_malyu

Why we would discuss memory models in Java here?

Only because the considerations apply to the C++ memory model (being the native memory model) as well.

If you are really writing code which should work on any hardware use fully compliant C++11 compiler.

Not all compilers are fully compliant, to put it mildly. But this is beside the point, I was just noting the usual problems encountered with global variables and singletons. Anyone is free to decide for themselves and use them or not (makes no difference to me).

Kind regards.