Please help: Download file from server using HTTP/2 protocol consumes too much RAM

kto_

Hi everyone,
I'm facing a very headache problem.
I use Qt 5.9.6 to write a simple program to download file from server which is using HTTP/2 protocol.
I observe that my program consumes amount of RAM same as the downloaded file size. If I download file 1GB then my RAM will increase gradually to 1GB and will be released when downloading finishes.

This issue does not occur if I download file from server using HTTP/1.1 protocol.

Here is my code:

    QNetworkAccessManager qnam;
    QNetworkReply *rep;

    // Open file to write downloaded data
    QFile file("C:\\Download\\downloaded.data");
    if (!file.open(QIODevice::WriteOnly)) {
        return;
    }

    // This server uses http/2 protocol
    QUrl url(QStringLiteral("https://nodejs.org/dist/v16.2.0/node-v16.2.0.tar.gz"));

    QNetworkRequest req(url);
    // Enable http/2 protocol for request.
    // If you download from server using http/1.1 please comment this line
    req.setAttribute(QNetworkRequest::HTTP2AllowedAttribute, true);

    // Start to download
    rep = qnam.get(req);

    // Connect signals
    QObject::connect(rep, &QNetworkReply::finished, [&](){
        // Download finished, stop event loop
        evtloop.exit();
    });
    QObject::connect(rep, &QNetworkReply::sslErrors, [&](const QList<QSslError> &errors){
        qDebug() << errors;
    });
    QObject::connect(rep, &QNetworkReply::readyRead, [&](){
        // Write data to file
        file.write(rep->readAll());
    });

    // Start event loop
    evtloop.exec();
    
    // Close file and delete NetworkReply pointer
    file.close();
    rep->deleteLater();

There're some reasons prevent me from upgrading to Qt's newer version so currently I'm still using Qt 5.9.6.

Please help me, I appreciate any help from you all.

Christian Ehrlicher

This post is deleted!

SGaist

Hi and welcome to devnet,

Since QNetworkReply is a QIODevice, why not use readyRead and read the data as they arrive ?

kto_

Hi @SGaist
Yes, I do use readyRead signal to read data when they arrive. You can check my code above

QObject::connect(rep, &QNetworkReply::readyRead, [&](){
        // Write data to file
        file.write(rep->readAll());
    });

SGaist

My bad, the code rendering is sometime tricky.

Even if locked with 5.9.6, can you check if it works better with a more recent version of Qt ?

kto_

Since all my projects are using Qt 5.9.6 so I cannot upgrade to newer version now (it will take time to make everything works fine with recent Qt version).
I wonder if we have a solution for this (or root cause is ok)

SGaist

The fact that you are locked to 5.9.6 is well understood, my suggestion is to check whether it's still the case with a more recent version of Qt and if it's the case, you might be able to backport said fix. You do not need to re-build your whole application, just make a minimal app that does the download to observe the effect.

kto_

Got it, let me try.
Thanks a lot for your help.

kto_

This post is deleted!

Christian Ehrlicher

It still would be nice if you can try to reproduce it with 5.15.x and if it still happens without you start/endTransaction() stuff, create a bug report so it gets fixed. I don't have a http/2 server for testing here so can't try it out by myself.

kto_

Sorry I forgot to enable HTTP/2. I re-checked again and RAM is still increased.
You can test by downloading file from here:
https://download-installer.cdn.mozilla.net/pub/firefox/releases/89.0/linux-x86_64/en-US/firefox-89.0.tar.bz2
This server is using HTTP/2 protocol.

Thuong Le

I've tried with 5.12 and the problem is not happening. Maybe it is a bug of 5.9.6