QByteArray not initializing
-
@mzimmers said in QByteArray not initializing:
@JNBarchan the pointer should be OK
What pointer, and what do you mean by "OK"?
P.S. I can explain why it goes through sometimes and fails others, but it's going to mean quite a bit of typing, do you really want to know?
@JNBarchan in this line of code:
sscanf(p, "%2x", (int *) &c);the address formed by the cast should be fine, as far as I can tell.
I agree the problem is in the sscanf. This line of code:
sscanf(p, "%hhx", &c);compiles but doesn't work right (c keeps getting 0xff). I'm going to go with aha_1980's approach and call it a day.
Thanks to everyone who looked at this.
-
@JNBarchan in this line of code:
sscanf(p, "%2x", (int *) &c);the address formed by the cast should be fine, as far as I can tell.
I agree the problem is in the sscanf. This line of code:
sscanf(p, "%hhx", &c);compiles but doesn't work right (c keeps getting 0xff). I'm going to go with aha_1980's approach and call it a day.
Thanks to everyone who looked at this.
@mzimmers said in QByteArray not initializing:
@JNBarchan in this line of code:
sscanf(p, "%2x", (int *) &c);the address formed by the cast should be fine, as far as I can tell.
The address is indeed a valid address, but it points to an area reserved to hold one byte and you're storing 4 or 8 bytes there, which is very naughty! Hope you understand.
-
@mzimmers said in QByteArray not initializing:
@JNBarchan in this line of code:
sscanf(p, "%2x", (int *) &c);the address formed by the cast should be fine, as far as I can tell.
The address is indeed a valid address, but it points to an area reserved to hold one byte and you're storing 4 or 8 bytes there, which is very naughty! Hope you understand.
@JNBarchan said in QByteArray not initializing:
which is very naughty!
Buffer overflows - the pinnacle of C programming ... ;)
Read and abide by the Qt Code of Conduct
-
@JNBarchan said in QByteArray not initializing:
which is very naughty!
Buffer overflows - the pinnacle of C programming ... ;)
-
@kshegunov I expected the "%2x" in the sscanf to restrict output to one byte (2 hex characters), but I guess it doesn't work that way.
Not really, no. There's no compile-time type checking with variadic functions (which
scanfand related are). It will expect you to give it a proper integer to write to. But really your error is the cast itself. You can't cast pointers like this. You have acharon the stack and you cast its address to aint *. From there on the compiler will consider that this pointer is referencing an integer, meaning you have "hijacked" 3 bytes from the nearby variables in the stack. If you modify your code like this:uint8_t c, c2, c3, c4; // ... sscanf(p, "%2x", (int *) &c); // ...You'd discover that all those variables -
c2,c3andc4are modified alongside your originalcbecausesscanfoverruns the given buffer and writes outside of it.Read and abide by the Qt Code of Conduct
-
The original code has:
QByteArray qMac; uint8_t c;My guess is
qMacis stored abovecin the local stack. When you write to&cyou cobble what's inqMac, so the bug doesn't show up till you reach theqMacloop. ThenqMac.append((char) c);gets evaluated and crashes. Am I right?Behaviour could vary across 32/64 size and specific compiler allocation or register code generated.
-
The original code has:
QByteArray qMac; uint8_t c;My guess is
qMacis stored abovecin the local stack. When you write to&cyou cobble what's inqMac, so the bug doesn't show up till you reach theqMacloop. ThenqMac.append((char) c);gets evaluated and crashes. Am I right?Behaviour could vary across 32/64 size and specific compiler allocation or register code generated.
-
Not really, no. There's no compile-time type checking with variadic functions (which
scanfand related are). It will expect you to give it a proper integer to write to. But really your error is the cast itself. You can't cast pointers like this. You have acharon the stack and you cast its address to aint *. From there on the compiler will consider that this pointer is referencing an integer, meaning you have "hijacked" 3 bytes from the nearby variables in the stack. If you modify your code like this:uint8_t c, c2, c3, c4; // ... sscanf(p, "%2x", (int *) &c); // ...You'd discover that all those variables -
c2,c3andc4are modified alongside your originalcbecausesscanfoverruns the given buffer and writes outside of it.@kshegunov said in QByteArray not initializing:
uint8_t c, c2, c3, c4; // ... sscanf(p, "%2x", (int *) &c); // ...You'd discover that all those variables -
c2,c3andc4are modified alongside your originalcbecausesscanfoverruns the given buffer and writes outside of it.@kshegunov Purely OOI, did you actually test this code? I was actually thinking, assuming local variables allocated down the stack in order they're encountered, you'd have to
sscanfintoc4rather thancto get that behaviour from the way they're laid out...? -
@kshegunov said in QByteArray not initializing:
uint8_t c, c2, c3, c4; // ... sscanf(p, "%2x", (int *) &c); // ...You'd discover that all those variables -
c2,c3andc4are modified alongside your originalcbecausesscanfoverruns the given buffer and writes outside of it.@kshegunov Purely OOI, did you actually test this code? I was actually thinking, assuming local variables allocated down the stack in order they're encountered, you'd have to
sscanfintoc4rather thancto get that behaviour from the way they're laid out...?@JNBarchan said in QByteArray not initializing:
@kshegunov Purely OOI, did you actually test this code?
Not at the time, no.
I was actually thinking, assuming local variables allocated down the stack in order they're encountered, you'd have to sscanf into c4 rather than c to get that behaviour from the way they're laid out...?
Depends on many things in fact, but most important of those would be the compiler (also which way the stack goes address-wise). Don't forget the compiler's allowed to reorder the variables and memory access freely (with exception of memory fences and
volatile, but that's another discussion) if it finds it convenient to do so. Meaning, if you don't reference a variable it may even strip it from the assembly. That is for release, though, I've noticed compilers (at least g++) will not remove things from the binary in debug mode even if redundant/unnecessary.Anyway, I just made a quick test with g++ (7.2):
uint8_t c = 0, c2 = 0, c3 = 0, c4 = 0; *reinterpret_cast<int *>(&c) = -1; // c = c2 = c3 = c4 = 0xFFand
uint8_t c = 0, c2 = 0, c3 = 0, c4 = 0, c5 = 0; *reinterpret_cast<int *>(&c) = -1; // c = c3 = c4 = c5 = 0xFF; c2 = 0x00and the assembly for the second one:
8 [1] uint8_t c = 0, c2 = 0, c3 = 0, c4 = 0, c5 = 0; 0x5555555547c5 <+0x000b> c6 45 fb 00 movb $0x0,-0x5(%rbp) 0x5555555547c9 <+0x000f> c6 45 ff 00 movb $0x0,-0x1(%rbp) 0x5555555547cd <+0x0013> c6 45 fe 00 movb $0x0,-0x2(%rbp) 0x5555555547d1 <+0x0017> c6 45 fd 00 movb $0x0,-0x3(%rbp) 0x5555555547d5 <+0x001b> c6 45 fc 00 movb $0x0,-0x4(%rbp) 9 [1] *reinterpret_cast<int *>(&c) = -1; 0x5555555547d9 <+0x001f> 48 8d 45 fb lea -0x5(%rbp),%rax 0x5555555547dd <+0x0023> c7 00 ff ff ff ff movl $0xffffffff,(%rax)As you can see the variable
cwas actually reordered by the compiler to occupy an address deeper in the stack.Read and abide by the Qt Code of Conduct
-
@JNBarchan said in QByteArray not initializing:
@kshegunov Purely OOI, did you actually test this code?
Not at the time, no.
I was actually thinking, assuming local variables allocated down the stack in order they're encountered, you'd have to sscanf into c4 rather than c to get that behaviour from the way they're laid out...?
Depends on many things in fact, but most important of those would be the compiler (also which way the stack goes address-wise). Don't forget the compiler's allowed to reorder the variables and memory access freely (with exception of memory fences and
volatile, but that's another discussion) if it finds it convenient to do so. Meaning, if you don't reference a variable it may even strip it from the assembly. That is for release, though, I've noticed compilers (at least g++) will not remove things from the binary in debug mode even if redundant/unnecessary.Anyway, I just made a quick test with g++ (7.2):
uint8_t c = 0, c2 = 0, c3 = 0, c4 = 0; *reinterpret_cast<int *>(&c) = -1; // c = c2 = c3 = c4 = 0xFFand
uint8_t c = 0, c2 = 0, c3 = 0, c4 = 0, c5 = 0; *reinterpret_cast<int *>(&c) = -1; // c = c3 = c4 = c5 = 0xFF; c2 = 0x00and the assembly for the second one:
8 [1] uint8_t c = 0, c2 = 0, c3 = 0, c4 = 0, c5 = 0; 0x5555555547c5 <+0x000b> c6 45 fb 00 movb $0x0,-0x5(%rbp) 0x5555555547c9 <+0x000f> c6 45 ff 00 movb $0x0,-0x1(%rbp) 0x5555555547cd <+0x0013> c6 45 fe 00 movb $0x0,-0x2(%rbp) 0x5555555547d1 <+0x0017> c6 45 fd 00 movb $0x0,-0x3(%rbp) 0x5555555547d5 <+0x001b> c6 45 fc 00 movb $0x0,-0x4(%rbp) 9 [1] *reinterpret_cast<int *>(&c) = -1; 0x5555555547d9 <+0x001f> 48 8d 45 fb lea -0x5(%rbp),%rax 0x5555555547dd <+0x0023> c7 00 ff ff ff ff movl $0xffffffff,(%rax)As you can see the variable
cwas actually reordered by the compiler to occupy an address deeper in the stack.@kshegunov you'll see reordering in other areas of code as well. For bit-bangers like me, the most notorious is the realignment of scalars within a structure. Some of the compilers for embedded systems have ways to prevent it, but you usually have to take care of it yourself.
In retrospect it's pretty obvious that sscanf would behave the way it did. At the time, I was just hustling through a problem and didn't think it through.
One of the challenges I'm having on this project is all the necessary data type conversions for data streams. As a C++ programmer, I'm partial to the string object, but I'm using an AES facility that likes uints, an authentication utility that uses char arrays, and then there's Qt's QStrings and QByteArrays. The compiler catches a lot of mismatches, but there are still a lot of ways to hit the weeds.
-
@kshegunov you'll see reordering in other areas of code as well. For bit-bangers like me, the most notorious is the realignment of scalars within a structure. Some of the compilers for embedded systems have ways to prevent it, but you usually have to take care of it yourself.
In retrospect it's pretty obvious that sscanf would behave the way it did. At the time, I was just hustling through a problem and didn't think it through.
One of the challenges I'm having on this project is all the necessary data type conversions for data streams. As a C++ programmer, I'm partial to the string object, but I'm using an AES facility that likes uints, an authentication utility that uses char arrays, and then there's Qt's QStrings and QByteArrays. The compiler catches a lot of mismatches, but there are still a lot of ways to hit the weeds.
@mzimmers said in QByteArray not initializing:
For bit-bangers like me, the most notorious is the realignment of scalars within a structure.
Do you mean structure padding? If so, that's quite known and well documented. And usually the compiler does that for a good reason, but this is not reordering as such. Here the compiler actually switched the memory layout of the variables for whatever reason, and you don't see that with structures, it may only include padding between the members but order is fixed.
Read and abide by the Qt Code of Conduct
-
@mzimmers said in QByteArray not initializing:
For bit-bangers like me, the most notorious is the realignment of scalars within a structure.
Do you mean structure padding? If so, that's quite known and well documented. And usually the compiler does that for a good reason, but this is not reordering as such. Here the compiler actually switched the memory layout of the variables for whatever reason, and you don't see that with structures, it may only include padding between the members but order is fixed.
-
@kshegunov I did a little research; it looks like the problem has been obviated in C++11:
Good job! I didn't know that particular peculiarity, simply because it never occurred to me one could use a struct/class with interleaving access specifiers for POD. Perhaps that's the reason I had never hit it and thus me not knowing it. Anyway, thanks for the info, it's appreciated!
-
Do you mean structure padding? If so, that's quite known and well documented.
In rare cases you may have to use the #pragma pack directive
https://stackoverflow.com/questions/3318410/pragma-pack-effect
