QByteArray and char type

J.Hilk · 25 Aug 2020, 10:42

@JonB clang is actually a bit more detailed

main.cpp:88:18: error: use of overloaded operator '>=' is ambiguous (with operand types 'QByteRef' and 'int')
qbytearray.h:554:17: note: candidate function
main.cpp:88:18: note: built-in candidate operator>=(int, int)
main.cpp:88:18: note: built-in candidate operator>=(float, int)
main.cpp:88:18: note: built-in candidate operator>=(double, int)
main.cpp:88:18: note: built-in candidate operator>=(long double, int)
main.cpp:88:18: note: built-in candidate operator>=(int, float)
main.cpp:88:18: note: built-in candidate operator>=(int, double)
...
main.cpp:88:18: note: built-in candidate operator>=(unsigned __int128, unsigned long)
main.cpp:88:18: note: built-in candidate operator>=(unsigned __int128, unsigned long long)
main.cpp:88:18: note: built-in candidate operator>=(unsigned __int128, unsigned __int128)

and it's true, the operator overload of [] for QByteArray returns either a QByteRef or a char so, its ambiguous

where as at() is guaranteed to be of the type char

also, I get an warning for the implicit conversion so 🤷‍♂️

make this if (b[0] >= 127) explicitly a char not an implicit int, and the warning should go away

if (b[0] >= char(127))

JonB · 25 Aug 2020, 10:48

@J-Hilk
Damn! A certain person (I'm looking at you, @mrjj :) ) told me to switch off Clang in Qt Creator and go for the editing experience without. (Partly, IIRC, because of Clang's ridiculous ordering of proposed completions for method names, which makes it awful to use.) So, like a lamb to the slaughter, I have followed his advice, and do not get that information about which overload of [] it was going for....

So, off topic, but: in view of this, would you, @J-Hilk, or others, advise me to revert to the default of Clang being on? :)

J.Hilk · 25 Aug 2020, 10:51

@JonB said in QByteArray and char type:

So, off topic, but: in view of this, would you, @J-Hilk, or others, advise me to revert to the default of Clang being on? :)

IMHO the inclusion/support for clang as greatly improved, since its first introduction.
In the beginning, I also turned it of, and for about a year or so it's on by default (for me). And for me I had more positive experience with it then (hardly any) negative ones.

So, turn it on:D

JonB · 25 Aug 2020, 11:03

@J-Hilk
Thanks for advice. This is all @mrjj's fault ;-)

Then I have a question (to which I suspect I already know the answer, unfortunately). I use method-name completion all the time. Without Clang on the suggestions are alphabetical, which is good to navigate. But with Clang (last time I looked, anyway) the order is "pseudo-random" ;-) Algorithm for ordering might make sense to a machine, but not to a human.... Do you not find this an issue?

J.Hilk · J JonB 25 Aug 2020, 10:51

@JonB it still does that, I'm not sure why and you could probably make your own plugin to sort it before showing if it really bothers you :D

But usually I know the beginning of the method so I type the first 2 letters which usually is enough to narrow the selection down to a hand full of options 😉

JKSH · 25 Aug 2020, 13:13

@JonB said in QByteArray and char type:

I am surprised Qt has chosen to call a signed char type QByteArray.

Be aware that char and signed char are different types in C++: https://stackoverflow.com/questions/436513/char-signed-char-char-unsigned-char . int is guaranteed to be signed but char is not!

For ARM CPUs, char is unsigned by default: https://developer.arm.com/documentation/dui0491/i/C-and-C---Implementation-Details/Character-sets-and-identifiers -- "The ARM ABI defines char as an unsigned byte, and this is the interpretation used by the C++ libraries supplied with the ARM compilation tools"

Seems Qt would have been better naming it QCharArray, since that is more accurate than QByteArray, by normal naming conventions.

I'd say @aha_1980's "QDataArray" name would work better than "QCharArray". To me, a "char array" is more related to historical text strings than binary data... and QByteArray is intended to be a container of binary data (i.e. bytes). I have no problems with its current name; I just treat the char as an implementation detail (albeit a leaky one)

This (or similar code) is the danger of the existing implementation being used by someone, unaware that they will not produce what is (presumably) the "expected" result, given that he assumes he is dealing with "bytes".

What is the meaning of doing an inequality comparison between a byte and a number? 127 is not a byte.

they could introduce, say, a byte() method to correspond to the current data() method, to return unsigned char * instead of char *.

I don't see much point in switching from char to unsigned char. If we're to initiate a switch, let's do things properly and switch to std::byte.

JonB · J JKSH 25 Aug 2020, 13:08

@JKSH said in QByteArray and char type:
I was not aware that char is no longer defined as signed (God bless C). Thank you for pointing that out.

What is the meaning of doing an inequality comparison between a byte and a number? 127 is not a byte.

Given the use of the word "byte" in QByteArray, I am (arrogantly) confident that since

    QByteArray b;
    b.resize(1);
    b[0] = 128;
    if (b.at(0) >= 127)
        qDebug() << "Yes";

goes through gcc without warning and does not produce "Yes" it will catch people out, if I could look through a whole bunch of people's code.... It's an observation. In part inspired from the confusion shown in the https://forum.qt.io/topic/118343/qbytearray-range-issue thread.

J.Hilk · 26 Aug 2020, 06:21

@JKSH said in QByteArray and char type:

I don't see much point in switching from char to unsigned char. If we're to initiate a switch, let's do things properly and switch to std::byte.

is someone(tm) where to make the changes, like @aha_1980 suggested in this bug report: https://bugreports.qt.io/browse/QTBUG-64746

you would prefer std::byte over unsigned char ?

Because Thiago was against scope creep, and adding std::byte and unsigned char probably falls in that category

that said, std::byte would make that Qt Version require c++17 or later. I'm not sure, that's ok, or not ?

aha_1980 · 26 Aug 2020, 06:26

Good morning @J-Hilk,

that said, std::byte would make that Qt Version require c++17 or later. I'm not sure, that's ok, or not ?

That is no problem, as Qt 6 requires C++17.

But as std::byte is also with limited scope (no arithmetic) I'm not sure it is a general solution...

J.Hilk · 26 Aug 2020, 06:42

@aha_1980 good morning to you too!

That is no problem, as Qt 6 requires C++17.

where did you get that from ? I spend like 30 min searching for any reference and didn't find anything :(

aha_1980 · J J.Hilk 26 Aug 2020, 06:26

Hi @J-Hilk ,

we've missed you at 2019 Contributers summit ;)

https://wiki.qt.io/Qt_Contributors_Summit_2019_Program#C.2B.2B17_language_and_std_library_features_for_Qt_6

JKSH · 26 Aug 2020, 08:43

@J-Hilk said in QByteArray and char type:

@JKSH said in QByteArray and char type:

I don't see much point in switching from char to unsigned char. If we're to initiate a switch, let's do things properly and switch to std::byte.

is someone(tm) where to make the changes, like @aha_1980 suggested in this bug report: https://bugreports.qt.io/browse/QTBUG-64746

you would prefer std::byte over unsigned char ?

Actually, I take that back. I just tried playing std::byte and found that it's not easy to work with:

std::byte b = 0xFF; // Error: cannot initialize a variable of type 'std::byte' with an rvalue of type 'int'

auto x = std::byte{0xFF};
auto y = uchar{0xFF};
qDebug() << (x == y); // Error: Invalid operands to binary expression

We also can't pass std::byte to a function that expects unsigned char without casting, so it isn't any more interoperable than the existing char.

Because Thiago was against scope creep, and adding std::byte and unsigned char probably falls in that category

I was originally thinking of adding functions that operate on std::byte and omitting functions that operate on unsigned char. I'm no longer convinced that's helpful.

std::byte would make that Qt Version require c++17 or later. I'm not sure, that's ok, or not ?

As @aha_1980 pointed out, this part isn't an issue.

The bigger issue is reaching a consensus on how far we should go:

Thiago Maciera wants to keep things as-is but is open to letting "someone(TM)" add a few convenience functions for interop with unsigned char.
Marc Mutz wants to rework QByteArray completely to use std::byte under the hood: https://lists.qt-project.org/pipermail/development/2020-May/039532.html

@aha_1980 said in QByteArray and char type:

Hi @J-Hilk ,

we've missed you at 2019 Contributers summit ;)

https://wiki.qt.io/Qt_Contributors_Summit_2019_Program#C.2B.2B17_language_and_std_library_features_for_Qt_6

There's also the blog post at https://www.qt.io/blog/first-qt-6.0-snapshot-available

P.S. Anyone signed up for the virtual Qt World Summit? :-D

J.Hilk · 26 Aug 2020, 08:59

@JKSH alright, lets see if that "someone(TM)" ends up being me. Have to contribute eventually 🤷‍♂️

and yes I bought a ticket already ;)

@aha_1980

we've missed you at 2019 Contributers summit ;)

I'm not a source code contributor (yet :) )

JonB · J JKSH 26 Aug 2020, 08:32

@JKSH said in QByteArray and char type:

Actually, I take that back. I just tried playing std::byte and found that it's not easy to work with:

You guys know more about C++ than I, but my reading of std::byte() is that it is effectively just a representation of an 8-bit pattern. You are not supposed to do any arithmetic on it, or natively compare it to unsigned char etc. It's just a "blob" of data. ?

JKSH · 26 Aug 2020, 09:06

@JonB said in QByteArray and char type:

my reading of std::byte() is that it is effectively just a representation of an 8-bit pattern.

I agree.

(Caveat: A byte is defined as the smallest accesible unit of data in memory. It's usually 8-bits in today's common architectures, but it doesn't actually have to be 8-bits)

You are not supposed to do any arithmetic on it

I agree. And I think programmers shouldn't normally try to do arithmetic on QByteArray elements either. (Exception: If you have a low-level efficiency hack in mind, you really know what you're doing, and you document it clearly, then go ahead)

...or natively compare it to unsigned char etc. It's just a "blob" of data. ?

Wasn't your original point of this thread that a "blob" of data should be unsigned char?

aha_1980 · 26 Aug 2020, 09:03

@J-Hilk said in QByteArray and char type:

I'm not a source code contributor (yet :) )

I don't think that's a precondition. You contribute on many other places.

And you have a good knowledge about the library and a vision where it should go to.

And that's what counts :)

Regards

JKSH · wrote on 26 Aug 2020, 09:03

@aha_1980 said in QByteArray and char type:

@J-Hilk said in QByteArray and char type:

I'm not a source code contributor (yet :) )

I don't think that's a precondition. You contribute on many other places.

And you have a good knowledge about the library and a vision where it should go to.

And that's what counts :)

+1 @J-Hilk is definitely a Contributor to the Qt community.

JonB · 26 Aug 2020, 11:19

@JKSH said in QByteArray and char type:

I agree. And I think programmers shouldn't try to do arithmetic on QByteArray elements either.

That's fine if I receive some QByteArray data and just want to store it/forward it onto something else. It's not fine if I need to look at its content and act on it for some purpose. Then I may need to, say, see if it's greater than 200 or whatever. At which point I think I need to cast away from std::byte() to achieve that.

Wasn't your original point of this thread that a "blob" of data should be unsigned char?

I was not the person who introduced the discussion about representing it via std::byte, for good or for bad! I want to be able to examine the bytes and do, for example, greater-then operations on them. For that, my original point was that I did not expect something referring to "bytes" --- using at least what I have found usage of that word in other languages to be, viz. an unsigned quantity in range 0--255 --- to have an interface only offering (signed) chars, I expected unsigned chars to be available. Else one must be careful about comparison code, for instance.

J.Hilk · 26 Aug 2020, 09:59

Let me bring even more confusion in this and point to Timur Doumler excellent talk at CppCon 2019 about type punning, where he outlines that this:

void printBitRepresentation(float f)
{
    auto buf* = reinterpret_cast<unsigned char*>(&f);
         for( int i(0); i < sizeof(float); i++ ) {
                std::cout << buf[i];
         }
}

is actually undefined behavior.
https://youtu.be/_qzMpk-22cc?t=2626

@JKSH thanks :D

JonB · 26 Aug 2020, 10:07

@J-Hilk
I did have a look at that (frightening) discussion. I was "perturbed" by the answer that you have to rely on what he said was a "magic" implementation of memcpy(), which you can't know anything about, to achieve it! And didn't really understand how that resolves whatever the issue is anyway.