Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Please help: Download file from server using HTTP/2 protocol consumes too much RAM



  • Hi everyone,
    I'm facing a very headache problem.
    I use Qt 5.9.6 to write a simple program to download file from server which is using HTTP/2 protocol.
    I observe that my program consumes amount of RAM same as the downloaded file size. If I download file 1GB then my RAM will increase gradually to 1GB and will be released when downloading finishes.

    This issue does not occur if I download file from server using HTTP/1.1 protocol.

    Here is my code:

        QNetworkAccessManager qnam;
        QNetworkReply *rep;
    
        // Open file to write downloaded data
        QFile file("C:\\Download\\downloaded.data");
        if (!file.open(QIODevice::WriteOnly)) {
            return;
        }
    
        // This server uses http/2 protocol
        QUrl url(QStringLiteral("https://nodejs.org/dist/v16.2.0/node-v16.2.0.tar.gz"));
    
        QNetworkRequest req(url);
        // Enable http/2 protocol for request.
        // If you download from server using http/1.1 please comment this line
        req.setAttribute(QNetworkRequest::HTTP2AllowedAttribute, true);
    
        // Start to download
        rep = qnam.get(req);
    
        // Connect signals
        QObject::connect(rep, &QNetworkReply::finished, [&](){
            // Download finished, stop event loop
            evtloop.exit();
        });
        QObject::connect(rep, &QNetworkReply::sslErrors, [&](const QList<QSslError> &errors){
            qDebug() << errors;
        });
        QObject::connect(rep, &QNetworkReply::readyRead, [&](){
            // Write data to file
            file.write(rep->readAll());
        });
    
        // Start event loop
        evtloop.exec();
        
        // Close file and delete NetworkReply pointer
        file.close();
        rep->deleteLater();
    

    There're some reasons prevent me from upgrading to Qt's newer version so currently I'm still using Qt 5.9.6.

    Please help me, I appreciate any help from you all.


  • Lifetime Qt Champion

    This post is deleted!

  • Lifetime Qt Champion

    Hi and welcome to devnet,

    Since QNetworkReply is a QIODevice, why not use readyRead and read the data as they arrive ?



  • Hi @SGaist
    Yes, I do use readyRead signal to read data when they arrive. You can check my code above

    QObject::connect(rep, &QNetworkReply::readyRead, [&](){
            // Write data to file
            file.write(rep->readAll());
        });
    

  • Lifetime Qt Champion

    My bad, the code rendering is sometime tricky.

    Even if locked with 5.9.6, can you check if it works better with a more recent version of Qt ?



  • Since all my projects are using Qt 5.9.6 so I cannot upgrade to newer version now (it will take time to make everything works fine with recent Qt version).
    I wonder if we have a solution for this (or root cause is ok)


  • Lifetime Qt Champion

    The fact that you are locked to 5.9.6 is well understood, my suggestion is to check whether it's still the case with a more recent version of Qt and if it's the case, you might be able to backport said fix. You do not need to re-build your whole application, just make a minimal app that does the download to observe the effect.



  • Got it, let me try.
    Thanks a lot for your help.



  • This post is deleted!

  • Lifetime Qt Champion

    It still would be nice if you can try to reproduce it with 5.15.x and if it still happens without you start/endTransaction() stuff, create a bug report so it gets fixed. I don't have a http/2 server for testing here so can't try it out by myself.



  • Sorry I forgot to enable HTTP/2. I re-checked again and RAM is still increased.
    You can test by downloading file from here:
    https://download-installer.cdn.mozilla.net/pub/firefox/releases/89.0/linux-x86_64/en-US/firefox-89.0.tar.bz2
    This server is using HTTP/2 protocol.


Log in to reply