Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Variables memory placement



  • Hi, I need to point some structures with a unique pointer, but I noted that the structures declared as continguous are not contiguous in memory and this doesn't surprise me. I know I can insert all that structures in an external structure, and in that case the placement is guaranteed, but I cannot do that for other reasons. Is there any qmake option (or someother trick) to force it to place in contiguous memory area the structures declared as contiguous ?
    Thanks in advance !
    Vittorio


  • Lifetime Qt Champion

    Hi
    How do you declare them as continuous?

    also, if using a unique pointer, why does that even matter, if they are continuous or not? (if i may ask)



  • Suppose I have ...
    struct
    {
    int var1;
    int var2;
    } myStruct_1;
    struct
    {
    int var1;
    int var2;
    } myStruct_2;
    ...
    struct
    {
    int var1;
    int var2;
    } myStruct_n;
    I prepared some custom widgets and, in the designer, I place them in the form and I set a property that represent the offset since the beginning of that area so, if the structures were contiguous, my handler could update that widgets just having a unique pointer, the address of the first structure. In my case, this works properly for the first structure, but fails with the following ones because the structures are not contiguous. This is just a simplification, but it's just to have an idea.


  • Lifetime Qt Champion

    @Vittorio61
    Hi
    Ok. and its not an option to put them into a std:.vector ?

    std::vector<struct> as that will guarantee they are continuous?

    Have you tried to allocate them in .cpp ?

    myStruct_1 t1;
    myStruct_2 t2;
    myStruct_3 t3;

    But then again, this won't guarantee anything and might explode later on.



  • Since op stated that a struct of structs is not an option, then all bets are off. Even then, the #pragma pack() would be necessary to guarantee packed alignment of the elements (no holes).

    A vector of structs would guarantee continuity but not packed alignment.



  • Firstly, I really don't think it's a good idea to try having separately-declared variables and then relying on their adjacent declarations in code causing adjacent layout. It's not very language-y, you are relying on compiler behaviour. Just not a good idea.

    Secondly, as it stands the OP cannot do

    if the structures were contiguous, my handler could update that widgets just having a unique pointer, the address of the first structure

    as simply as he implies. As written, these are separate struct types. That means they could have different member types, so you cannot assume one struct is the same size as another, and therefore you cannot "march through them" with a fixed size.

    It's important the OP states whether these structs are actually all the same struct type (as they are in the code he shows) or not. In which case he should change to use the same struct type.

    An array is the only way to safely guarantee layout order. If they are all the same type he can have an array of them. If they are not, then he would have to have an array of unions of all the struct types. You can then march through them via the array index, they are adjacent, and FWIW each one occupies &array[1] - &array[0] bytes.

    Another way to "march through" of course is, after the variables are defined, to have an array of pointers to initialized to point to each of the variables, and use that to address them in a loop.



  • Thanks for your replies. Despite of the examples the structures are very different one from the others. I tried with #pragma and it works as expected for the variables in a structure, but not between the structures. I checked the addressess: the following structures seems to be always aligned module 8. I'll have some other tests and if this is the only problem I will align my structures with fillers. In that case I'll consider this topic as "solved".


  • Lifetime Qt Champion

    @Vittorio61 said in Variables memory placement:

    I checked the addressess: the following structures seems to be always aligned module 8.

    since you're on 64bit, the alignment is 8bytes (64bits)
    But the order is not defined by definiton. So you need to create a big struct with proper pack and fill bytes.



  • @Vittorio61
    As I and @Christian-Ehrlicher said

    But the order is not defined by definiton.

    So your concept may be relying on compiler/platform implementation, and may not hold elsewhere.... Up to you.



  • Thanks to all, I wil take up all your suggestions and try to adapt my software accordingly.


  • Moderators

    @Vittorio61 said in Variables memory placement:

    I tried with #pragma and it works as expected for the variables in a structure, but not between the structures

    Like others mentioned you'd have to pack the stuctures and the class that houses the variables, so e.g.

    #pragma pack(1)
    class Foo
    {
       #pragma pack(1)
       struct
       {
       ...
       } myStruct_1;
    
       #pragma pack(1)
       struct
       {
       ...
       } myStruct_2;
    };
    

    Keep in mind that alignment is there for a reason and packing it this way hinders the performance of the class, because all access to it becomes unaligned and thus more costly.


  • Lifetime Qt Champion

    @Chris-Kawa This will still lead to an alignment of 4 or 8 for myStruct_2 which may not be what the op wants.


  • Moderators

    @Christian-Ehrlicher said:

    This will still lead to an alignment of 4 or 8 for myStruct_2

    It shouldn't. The packing of the outer class should take care of that. It does at least on MSVC: https://godbolt.org/z/4vvPxY



  • we're getting into very compiler/platform specific features with trying to stuff the data and using pragmas to do it. I've learned to avoid it altogether and to defined memory regions as simple uint8_t vectors, then to write accessor methods that push and pull the data by brute force and bit shifting/stuffing. I must do this a lot with ad-hoc networking protocols. One thing you never want to do is

    int v =*(int*)&(vec[17]);
    

    at the minimum it creates an innefficient memory access, and at the maximum generates an alignment fault on some processors. The better way is:

    int v;
    memcpy(&v, &vec[17], sizeof(int));
    

    manually setting significant bit order if necessary


Log in to reply