Howto neglect these bytes while downloading file from http server

pingal

I'm downloading binary file(s) from a python http server.
The downloading is fine but I get 4 extra bytes appended to my binary file every time I receive a file. I believe these bytes represents time. Although, I can simply strip these bytes within my program but what if in future I use this program with another server and that server transfer some more/less bytes along with the requested file (?)

Is their a generic way of how to get only the requested file bytes ?

Here is how I'm receiving file

QNetworkRequest request;
    request.setUrl(QUrl("http://link/to/file"));

void HTTP_Class::GetReq(QNetworkRequest &request){

    connect(&manager, &QNetworkAccessManager::finished, this, &HTTP_Class::replyFinished);
    reply = manager.get(request);

}

void HTTP_Class::replyFinished(){

    qDebug() << "replyFinished is called";
    QByteArray bytes;

    if (reply->error() == QNetworkReply::NoError){
        bytes = reply->readAll();
    }
    QFile file("path/to/file");
    if (!file.open(QIODevice::WriteOnly))
        return;

    QDataStream out(&file);
    out << bytes;
    file.close();
}

Here is an example which shows extra bytes along with the text "testing"

~~00 00 00 07~~ 74 65 73 74 69 6E 67 // testing

jsulm

@pingal said in Howto neglect these bytes while downloading file from http server:

Is their a generic way of how to get only the requested file bytes ?

You need to know the format of the data you're downloading. Qt can't know what the data you're downloading mean. So clarify the format and handle it accordingly.

ChrisW67

The four bytes look like the number of bytes in the remainder of the message.

pingal

ChrisW67: You are right. That's the data length. The problem was in the below lines:

QDataStream out(&file);
out << bytes;

When bytes are written to QDataStream in this fashion, Its write the content length followed by the actual data. (I thought it will just write the content).

So i figured out to use QDataStream::writeRawData

QDataStream out(&file);
 size_t size = bytes.size();
 out.writeRawData(bytes, size);
 file.close();

Christian Ehrlicher

Why do you need a QDataStream then at all?

pingal

I don't actually need it if i just use it for downloading a file, In fact, it is easier to just use QFile::write(..) in the above case. I though it would be easier to prefix/suffix files content like below:

out << prefix;
out << file;
out << suffix

Christian Ehrlicher

If you use QDataStream on the sender side you must also use it on the receiver side. Everything else will break sooner or later.

pingal

@Christian-Ehrlicher : I didn't notice that, can you please elaborate it a little.

Christian Ehrlicher

Don't know what else I should say - you use QDataStream to create your stream so you have to use QDataStream to deserialize the data later on. QDataStream adds data to the stream (as you already noticed) and may alter your data in any way: "A data stream is a binary stream of encoded information which is 100% independent of the host computer's operating system, CPU or byte order. For example, a data stream that is written by a PC under Windows can be read by a Sun SPARC running Solaris."
So reading it as raw data may or may not work. The format of QDataStream's binary representation is not documented so you it can change.

pingal

oky. thank you :)

ChrisW67

@pingal I missed that in your original post. I assumed you were receiving extra bytes, which is not the case.

pingal

@ChrisW67 Yes, that's what i thought at first but when I STDOUT QByteArray, then i realized that serializing data to stream is causing those extra bytes.