Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

How to write directly in to a QByteArray held by a QVariant



  • I have a QVariant that is used to hold some arbitrary data.
    In one use case, I need it to hold an array of 1000 bytes. So I make the QVariant hold a QByteArray.

    I have another function, that reads new data from some source in to a regular char array of 1000 bytes.
    When new data comes in, I need to update my QVariant to hold the new data.

    One simple way would be:

    char new_data[1000];
    QVariant v;
    ...
    void getNewData()
    {
       //read data in to new_data
       ...
       v.setValue( QByteArray(new_data, 1000) );
    }
    

    getNewData() is called in whenever new data is available from the source.

    But I am concerned about how many memory operations this would result in. I'm assuming that:

    • creation of QByteArray temporary will cause a memcpy from read_data
    • the old QByteArray held by the variant will be deallocated
    • another memcpy to copy the QByteArray in to the variant's storage mechanism (which might be avoided by some move operation, I'm not sure though).

    I feel I can manage with just one memcpy if I could access the QByteArray held by the QVariant and just write in to its buffer (QByteArray::data()).

    Is there any way to do this? Or some other mechanism that would be the most optimal?


  • Lifetime Qt Champion

    Moving a QByteArray into QVariant doesn't cost much. But the old one is deallocated, yes. There is no chance to avoid this.
    The memcpy can be avoided by directly filling the data into QByteArray instead using the temporary new_data[] allocation.



  • @Christian-Ehrlicher Thank you. I cannot read new data directly in to a QByteArray as that part of the code is from somewhere else and I have to work with a regular char*.
    So, I take it that there is no way to access the QByteArray held by the QVariant so that I can directly memcpy in to it?


  • Lifetime Qt Champion

    @syamcr said in How to write directly in to a QByteArray held by a QVariant:

    and I have to work with a regular char*.

    I don't understand this part. Please take a look in the documentation to see what a QByteArray is.



  • @Christian-Ehrlicher Sorry for the confusion. I know a QByteArray has an internal char array and I can use the data() member to access it and write in to it. What I meant is that for some other reasons, I get the data by means of a char* pointer. I am free to use it any way I want (create a QByteArray from it, for example), but I can't change the part of the code that gets the data in to that char buffer in the first place to use a QByteArray instead.



  • @syamcr said in How to write directly in to a QByteArray held by a QVariant:

    So, I take it that there is no way to access the QByteArray held by the QVariant so that I can directly memcpy in to it?

    @Christian-Ehrlicher would have to comment if this is correct, but it seems to me the QByteArray QVariant::toByteArray() const returns the variant's QByteArray just as much as if you want to access it outside of a QVariant?


  • Lifetime Qt Champion

    So your example was wrong :)



  • @Christian-Ehrlicher In the example, there's a comment:

    //read data in to new_data
    

    That represents the process that gets the data in to a char array. And I can't change it to read in to a QByteArray, for some other reasons. I hope it's clear now. The example code is not identical to my actual code.



  • @JonB QVariant::toByteArray() will get me a copy of its QByteArray and memcpy-ing in to it will not change the QVariant's data.


  • Lifetime Qt Champion

    @syamcr said in How to write directly in to a QByteArray held by a QVariant:

    That represents the process that gets the data in to a char array. And I can't change it to read in to a QByteArray, for some other reasons

    I don't see a difference passing a char* pointer from new_data or from QByteArray... so no I don't understand the problem.



  • @syamcr
    I believe you intend to go memcpy(byteArray.data(), ...), to write into the data buffer, is that right? I believe that (char *QByteArray::data()) causes a deep copy to occur, won't that mess up your whole desire to do everything in the original passed in char new_data[1000]? But @Christian-Ehrlicher knows what's he's talking about more than I do :)


  • Moderators

    Use QByteArray::fromRawData to attach a byte array to the existing buffer without making a copy of the data.

    const QByteArray data = QByteArray::fromRawData(new_data, 1000);
    v.setValue(data);
    

    @Christian-Ehrlicher said in How to write directly in to a QByteArray held by a QVariant:

    I don't see a difference passing a char* pointer from new_data or from QByteArray... so no I don't understand the problem.

    The library (or whatever it is) may allocate the block itself and just give you back the buffer to work with, so no passing of output pointer ... (I've seen such APIs, abysmal they may be).



  • @JonB said in How to write directly in to a QByteArray held by a QVariant:

    I believe that (char *QByteArray::data()) causes a deep copy to occur, won't that mess up your whole desire to do everything in the original passed in char new_data[1000]? But @Christian-Ehrlicher knows what's he's talking about more than I do :)

    I don't think just calling data() causes a deep copy, as the documentation says that we can change the QByteArray's data using this pointer. It might cause a deep copy if the QByteArray object was implicitly sharing its data with another object, or may be if the data had been set using QByteArray::setRawData().

    @kshegunov said in How to write directly in to a QByteArray held by a QVariant:

    Use QByteArray::fromRawData to attach a byte array to the existing buffer without making a copy of the data.

    Thank you. But my requirement is slightly different. I want a deep copy from my char array, as the char array could change later before the copied data is used. My QVariant is already holding a byte array. I want to memcpy directly to that object, so that overall there's only one memcpy operation and no allocation or deallocation of memory (other than the first time a byte array is assigned to the variant).

    The library (or whatever it is) may allocate the block itself and just give you back the buffer to work with, so no passing of output pointer ... (I've seen such APIs, abysmal they may be).

    Yes! Another possibility is if the char array is actually a slice of some bigger array.


  • Moderators

    @syamcr said in How to write directly in to a QByteArray held by a QVariant:

    I want a deep copy from my char array, as the char array could change later before the copied data is used.

    The QVariant is going to do a copy of the data, and that is inevitable.

    My QVariant is already holding a byte array. I want to memcpy directly to that object

    What you want is impossible.

    so that overall there's only one memcpy operation and no allocation or deallocation of memory (other than the first time a byte array is assigned to the variant).

    That can't be accomplished. The variant shall not (and must not) expose you its internal implementation.



  • @kshegunov @Christian-Ehrlicher Thank you both for all the information. I guess I will just do something like:

    v.setValue( QByteArray(new_data, 1000) );
    

    and not worry too much about avoiding memory allocations.


Log in to reply