Save data to a temporary file and clear it from RAM

[Sucharek]

Hi, I'm trying to make a downloader.
I got stuck on trying to save the current download progress to a temporary file and clear it from RAM, because it would be an issue when downloading big files.
I'm not sure how to do that. I tried to use deleteLater(), but that just freezes the download. I use that to clear it from RAM when the file is already downloaded. It works that way, but not here.

Here's what I have:
download.cpp

#include "download.h"

QElapsedTimer timer;
Downloader::Downloader(QObject *parent) : QObject(parent)
{
    manager = new QNetworkAccessManager();
    connect(manager, &QNetworkAccessManager::finished, this, &Downloader::onResult);

    timer.restart();
}

void Downloader::getData()
{
    QUrl url(link);
    QNetworkRequest request;
    request.setUrl(url);
    qDebug() << "Download...";
    reply = manager->get(request);
    connect(reply, &QNetworkReply::downloadProgress, this, &Downloader::onDownloadProgress);
    connect(reply, &QNetworkReply::readyRead, this, &Downloader::saveToFile);
}

qint64 previous;
double dlSpeed;
void Downloader::onDownloadProgress(qint64 bytesReceived, qint64 bytesTotal)
{
    if (timer.elapsed() == 0) {timer.start();}
    if (timer.elapsed() >= 1000) {
        double change = bytesReceived - previous;
        dlSpeed = change / 1000;
        previous = bytesReceived;
        timer.restart();
    }
    double progress = static_cast<qreal>(bytesReceived) / bytesTotal * 100.0;

    QString details = QString::number(bytesReceived / 1000) + " KB/" + QString::number(bytesTotal / 1000) + " KB "
                      "(" + QString::number(dlSpeed) + " KB/s)";
    qDebug() << "Download progress:" << details;
    qDebug() << "Download speed:" << dlSpeed << "KB/S";
    emit sendDetails(details);
    emit sendProgress(progress);
}

void Downloader::saveToFile() //here is where I'm trying to save it to a temporary file; it's not done yet, I was just trying to clear the current progress from memory
{
    QByteArray part = reply->readAll();
    qDebug() << part.size();
//    reply->deleteLater();
}

void Downloader::onResult(QNetworkReply *reply)
{
    if (reply->error()) {
        qDebug() << "ERROR";
        qDebug() << reply->errorString();
    } else {
        QString name = QFileInfo(link).fileName();
        QFile *file = new QFile("C:/Users/hi/Desktop/files/" + name);
        if (file->open(QFile::WriteOnly)) {
            file->resize(0);
            file->write(reply->readAll());
            file->close();

            qDebug() << "Download complete";
            reply->deleteLater();
        }
    }
}

download.h

#ifndef DOWNLOADER_H
#define DOWNLOADER_H

#include <QObject>
#include <QNetworkAccessManager>
#include <QNetworkRequest>
#include <QNetworkReply>
#include <QUrl>
#include <QFile>
#include <QDebug>
#include <QElapsedTimer>
#include <QFileInfo>

class Downloader : public QObject
{
    Q_OBJECT
public:
    explicit Downloader(QObject *parent = 0);

    void receiveLink(QString url) {link = url;}

signals:
    void sendDetails(QString);
    void sendProgress(int);

public slots:
    void getData();
    void onResult(QNetworkReply *reply);
    void onDownloadProgress(qint64 bytesReceived, qint64 bytesTotal);
    void saveToFile();

private:
    QNetworkAccessManager *manager;
    QNetworkReply *reply;

    QString link;
};

#endif // DOWNLOADER_H

Could you help me please?

[SGaist]

Did you check QNetworkReply's read buffer size ?

[SGaist]

Hi,

What exactly is your issue ?

You can open a file, append the data and then close it, or open the file when the download starts, write each block of data as they arrive to the file and when it's done close it.

[Sucharek]

Hi @SGaist, my problem is that when you're downloading a bigger file (like 10gb or more), your RAM can fill up and the download can fail.

[SGaist]

Did you check QNetworkReply's read buffer size ?

[Christian Ehrlicher]

Since QNetworkReply is a QIODevice you can connect to it's https://doc.qt.io/qt-6/qiodevice.html#readyRead signal and save the data to a file before the request is finished.

Since QNetworkReply is a QIODevice you can connect to it's https://doc.qt.io/qt-6/qiodevice.html#readyRead signal and save the data to a file before the request is finished.

[Sucharek]

@SGaist, sorry for the late reply.

That is exatcly what I was looking for. Combining the readyRead() signal and buffer size limit allowed me to save data to a temporaray file without using a lot of memory. Thanks.