Please help: Download file from server using HTTP/2 protocol consumes too much RAM

[kto_]

Hi everyone,
I'm facing a very headache problem.
I use Qt 5.9.6 to write a simple program to download file from server which is using HTTP/2 protocol.
I observe that my program consumes amount of RAM same as the downloaded file size. If I download file 1GB then my RAM will increase gradually to 1GB and will be released when downloading finishes.

This issue does not occur if I download file from server using HTTP/1.1 protocol.

Here is my code:

    QNetworkAccessManager qnam;
    QNetworkReply *rep;

    // Open file to write downloaded data
    QFile file("C:\\Download\\downloaded.data");
    if (!file.open(QIODevice::WriteOnly)) {
        return;
    }

    // This server uses http/2 protocol
    QUrl url(QStringLiteral("https://nodejs.org/dist/v16.2.0/node-v16.2.0.tar.gz"));

    QNetworkRequest req(url);
    // Enable http/2 protocol for request.
    // If you download from server using http/1.1 please comment this line
    req.setAttribute(QNetworkRequest::HTTP2AllowedAttribute, true);

    // Start to download
    rep = qnam.get(req);

    // Connect signals
    QObject::connect(rep, &QNetworkReply::finished, [&](){
        // Download finished, stop event loop
        evtloop.exit();
    });
    QObject::connect(rep, &QNetworkReply::sslErrors, [&](const QList<QSslError> &errors){
        qDebug() << errors;
    });
    QObject::connect(rep, &QNetworkReply::readyRead, [&](){
        // Write data to file
        file.write(rep->readAll());
    });

    // Start event loop
    evtloop.exec();
    
    // Close file and delete NetworkReply pointer
    file.close();
    rep->deleteLater();

There're some reasons prevent me from upgrading to Qt's newer version so currently I'm still using Qt 5.9.6.

Please help me, I appreciate any help from you all.

[Christian Ehrlicher]

This post is deleted!

[SGaist]

Hi and welcome to devnet,

Since QNetworkReply is a QIODevice, why not use readyRead and read the data as they arrive ?

[kto_]

Hi @SGaist
Yes, I do use readyRead signal to read data when they arrive. You can check my code above

QObject::connect(rep, &QNetworkReply::readyRead, [&](){
        // Write data to file
        file.write(rep->readAll());
    });

[SGaist]

My bad, the code rendering is sometime tricky.

Even if locked with 5.9.6, can you check if it works better with a more recent version of Qt ?

[kto_]

Since all my projects are using Qt 5.9.6 so I cannot upgrade to newer version now (it will take time to make everything works fine with recent Qt version).
I wonder if we have a solution for this (or root cause is ok)

[SGaist]

The fact that you are locked to 5.9.6 is well understood, my suggestion is to check whether it's still the case with a more recent version of Qt and if it's the case, you might be able to backport said fix. You do not need to re-build your whole application, just make a minimal app that does the download to observe the effect.

[kto_]

Got it, let me try.
Thanks a lot for your help.

[kto_]

This post is deleted!

[Christian Ehrlicher]

It still would be nice if you can try to reproduce it with 5.15.x and if it still happens without you start/endTransaction() stuff, create a bug report so it gets fixed. I don't have a http/2 server for testing here so can't try it out by myself.

[kto_]

Sorry I forgot to enable HTTP/2. I re-checked again and RAM is still increased.
You can test by downloading file from here:
https://download-installer.cdn.mozilla.net/pub/firefox/releases/89.0/linux-x86_64/en-US/firefox-89.0.tar.bz2
This server is using HTTP/2 protocol.

[Thuong Le]

I've tried with 5.12 and the problem is not happening. Maybe it is a bug of 5.9.6