Variables memory placement

Vittorio61

Hi, I need to point some structures with a unique pointer, but I noted that the structures declared as continguous are not contiguous in memory and this doesn't surprise me. I know I can insert all that structures in an external structure, and in that case the placement is guaranteed, but I cannot do that for other reasons. Is there any qmake option (or someother trick) to force it to place in contiguous memory area the structures declared as contiguous ?
Thanks in advance !
Vittorio

mrjj

Hi
How do you declare them as continuous?

also, if using a unique pointer, why does that even matter, if they are continuous or not? (if i may ask)

Vittorio61

Suppose I have ...
struct
{
int var1;
int var2;
} myStruct_1;
struct
{
int var1;
int var2;
} myStruct_2;
...
struct
{
int var1;
int var2;
} myStruct_n;
I prepared some custom widgets and, in the designer, I place them in the form and I set a property that represent the offset since the beginning of that area so, if the structures were contiguous, my handler could update that widgets just having a unique pointer, the address of the first structure. In my case, this works properly for the first structure, but fails with the following ones because the structures are not contiguous. This is just a simplification, but it's just to have an idea.

mrjj

@Vittorio61
Hi
Ok. and its not an option to put them into a std:.vector ?

std::vector<struct> as that will guarantee they are continuous?

Have you tried to allocate them in .cpp ?

myStruct_1 t1;
myStruct_2 t2;
myStruct_3 t3;

But then again, this won't guarantee anything and might explode later on.

Kent-Dorfman

Since op stated that a struct of structs is not an option, then all bets are off. Even then, the #pragma pack() would be necessary to guarantee packed alignment of the elements (no holes).

A vector of structs would guarantee continuity but not packed alignment.

JonB

Firstly, I really don't think it's a good idea to try having separately-declared variables and then relying on their adjacent declarations in code causing adjacent layout. It's not very language-y, you are relying on compiler behaviour. Just not a good idea.

Secondly, as it stands the OP cannot do

if the structures were contiguous, my handler could update that widgets just having a unique pointer, the address of the first structure

as simply as he implies. As written, these are separate struct types. That means they could have different member types, so you cannot assume one struct is the same size as another, and therefore you cannot "march through them" with a fixed size.

It's important the OP states whether these structs are actually all the same struct type (as they are in the code he shows) or not. In which case he should change to use the same struct type.

An array is the only way to safely guarantee layout order. If they are all the same type he can have an array of them. If they are not, then he would have to have an array of unions of all the struct types. You can then march through them via the array index, they are adjacent, and FWIW each one occupies &array[1] - &array[0] bytes.

Another way to "march through" of course is, after the variables are defined, to have an array of pointers to initialized to point to each of the variables, and use that to address them in a loop.

Vittorio61

Thanks for your replies. Despite of the examples the structures are very different one from the others. I tried with #pragma and it works as expected for the variables in a structure, but not between the structures. I checked the addressess: the following structures seems to be always aligned module 8. I'll have some other tests and if this is the only problem I will align my structures with fillers. In that case I'll consider this topic as "solved".

Christian Ehrlicher

@Vittorio61 said in Variables memory placement:

I checked the addressess: the following structures seems to be always aligned module 8.

since you're on 64bit, the alignment is 8bytes (64bits)
But the order is not defined by definiton. So you need to create a big struct with proper pack and fill bytes.

JonB

@Vittorio61
As I and @Christian-Ehrlicher said

But the order is not defined by definiton.

So your concept may be relying on compiler/platform implementation, and may not hold elsewhere.... Up to you.

Vittorio61

Thanks to all, I wil take up all your suggestions and try to adapt my software accordingly.

Chris Kawa

@Vittorio61 said in Variables memory placement:

I tried with #pragma and it works as expected for the variables in a structure, but not between the structures

Like others mentioned you'd have to pack the stuctures and the class that houses the variables, so e.g.

#pragma pack(1)
class Foo
{
   #pragma pack(1)
   struct
   {
   ...
   } myStruct_1;

   #pragma pack(1)
   struct
   {
   ...
   } myStruct_2;
};

Keep in mind that alignment is there for a reason and packing it this way hinders the performance of the class, because all access to it becomes unaligned and thus more costly.

Christian Ehrlicher

@Chris-Kawa This will still lead to an alignment of 4 or 8 for myStruct_2 which may not be what the op wants.

Chris Kawa

@Christian-Ehrlicher said:

This will still lead to an alignment of 4 or 8 for myStruct_2

It shouldn't. The packing of the outer class should take care of that. It does at least on MSVC: https://godbolt.org/z/4vvPxY

Kent-Dorfman

we're getting into very compiler/platform specific features with trying to stuff the data and using pragmas to do it. I've learned to avoid it altogether and to defined memory regions as simple uint8_t vectors, then to write accessor methods that push and pull the data by brute force and bit shifting/stuffing. I must do this a lot with ad-hoc networking protocols. One thing you never want to do is

int v =*(int*)&(vec[17]);

at the minimum it creates an innefficient memory access, and at the maximum generates an alignment fault on some processors. The better way is:

int v;
memcpy(&v, &vec[17], sizeof(int));

manually setting significant bit order if necessary