How to use QNetworkAccessManager without leaking memory.

Q139

Problem is similar to discussed here: [https://forum.qt.io/topic/93081/memory-leak-on-qnetworkaccessmanager](forum link)

Basically problem is with html requests growing memory usage, using QNetworkAccessManager.

.h

QNetworkAccessManager qnam;

.cpp , looped part.


reply = qnam.get(QNetworkRequest(url));

    connect(reply, &QNetworkReply::finished, this, &Form::httpFinished);

Then on QNetworkReply *reply; reply->deleteLater.

What am i doing wrong here that causes the leak?

So far i tested that replys deletelater wont fail.
If problem is with memory buildup in QNetworkAccessManager...
In the linked thread the solution is to delete QNetworkAccessManager ,but is there a way to clear individual component that grows?

I thought maybe delete QNetworkAccessManager after every 100 requests , so it can leak for while and then recreate.
But hopefully someone knows what is best to do.

SGaist

Hi,

What version of Qt are you using ?
On what platform ?
Can you provide a minimal compilable example that shows that behaviour ?

Q139

@SGaist

Qt Creator 4.11.1
Based on Qt 5.14.1 (MSVC 2017, 32 bit)

Built on Feb 5 2020 10:36:21

Windows 10, win 7 also

Let me know what needs to be cleaned there to avoid memory growth.
Or if some timeout needs to be set for AcessManager

CloseToMinimalExample

Q139

Sorry , forgot to add QApplication::processEvents(); inside while(1) loop , it may not process events at all, but with that added it grows slower
Also no valid adress as it may behave like ddos attack.

better .cpp file

#include "form.h"
#include "ui_form.h"
#include <QtNetwork>
#include <QUrl>
#include "QProgressDialog"
#include "QMessageBox"
#include "QTextBlock"
#include "qcoreapplication.h"
Form::Form(QWidget *parent) :
    QWidget(parent),
    ui(new Ui::Form)
{
    ui->setupUi(this);


#ifndef QT_NO_SSL
    connect(&qnam, &QNetworkAccessManager::sslErrors,
            this, &Form::sslErrors);
#endif

    ui->pushButton->click();

}

Form::~Form()
{
    delete ui;
}


bool finished=0;
void Form::on_pushButton_clicked()
{

ui->pushButton->hide();

   connect(reply, &QNetworkReply::finished, this, &Form::httpFinished);

while(1){

    QUrl ur("test");
    startRequest(ur);
    finished=0;

   while(!finished){
       QThread::msleep(1);
       QApplication::processEvents();
   }

}

}


void Form::startRequest(const QUrl &requestedUrl)
{
    reply = qnam.get(QNetworkRequest(url));

}

void Form::httpFinished()
{
    reply->deleteLater();
    finished=1;
 //  reply=NULL;
}

Is it safe to connect reply signal once and change pointers during runtime?
Is it safe and continues to work if at some point signals&slots sees signal as nullptr in eventloop while processing signals?

Q139

Here it says i should use clearConnectionCache();

clearAccessCache();
clearConnectionCache();

I tryed both cacheClears but it didnt help.
If added here:

void Form::startRequest(const QUrl &requestedUrl)
{
    qnam.clearAccessCache();
    qnam.clearConnectionCache();

    reply = qnam.get(QNetworkRequest(url));

}

Christian Ehrlicher

Please provide a minimal, compilable example without blocking the ui with a custom event loop.

Q139

@Christian-Ehrlicher There you go

Christian Ehrlicher

@Q139 This example contains a custom local eventloop. And this loop is not stopped due to while(1) around it. This is plain wrong.
Then you create a new request every millisecond and don't delete it.

Q139

@Christian-Ehrlicher How to delete request?

void Form::httpFinished()
{
    reply->deleteLater();
    finished=1;
 //  reply=NULL;
}

It should catch this function via event loop every time before new request, here it sets reply for deletion.

Christian Ehrlicher

@Q139 Ah, the strang finished=1 does it.
Again: don't block, use async signals/slots.

Your deleteLater() is the problem since you never return to the event loop. See the documentation: "The object will be deleted when control returns to the event loop. "
Creating a new eventloop will not help - it's a new one, not the one where the object was created in.

Q139

@Christian-Ehrlicher Does QApplication::processEvents(); not go trough deleteLater()'s event loop?

while(!finished){
    QThread::msleep(1);
    QApplication::processEvents();<--
}

At least it fires signals&slots events.

Christian Ehrlicher

@Q139 said in How to use QNetworkAccessManager without leaking memory.:

At least it fires signals&slots events.

It executes the event loop, it does not return to it. It would be dangerous to delete objects during processEvents() since you can still be in a slot which object was deferred for deletion.

Again: you don't need it and it's dangerous to call processEvents() or spin a local event loop. Avoid it as much as possible.

Q139

@Christian-Ehrlicher Then its solved, i tryed to bodge together networking by not returning from the while loop.
Assumption was that QApplication.processEvents() runs all functions of event loop including deletions.
It was for temprorary app i bodged together to collect data via certain api, but there is alot to get and many requests eat memory up.
Even bodging stuff together requires correct knowledge of how underliying functions works...

Is there a way to force deletion events from the while(1) event loop?

void Form::httpFinished()
{
    disconnect(reply, &QNetworkReply::finished, this, &Form::httpFinished);
    delete reply;
    finished=1;
 //  reply=NULL;
}

using delete reply; still grows memory

Q139

Without while loop i still notice memory usage growth , what am i missing here?
However it jumps back in memory usage after some periods, but still goes 100+mb eventualy.

#include "form.h"
#include "ui_form.h"
#include <QtNetwork>
#include <QUrl>
#include "QProgressDialog"
#include "QMessageBox"
#include "QTextBlock"
#include "qcoreapplication.h"
Form::Form(QWidget *parent) :
    QWidget(parent),
    ui(new Ui::Form)
{
    ui->setupUi(this);

    connect(this,SIGNAL(nextReq()),this,SLOT(startRequest()));//<--- to loop


#ifndef QT_NO_SSL
    connect(&qnam, &QNetworkAccessManager::sslErrors,
            this, &Form::sslErrors);
#endif

}

Form::~Form()
{
    delete ui;
}


bool finished=0;
void Form::on_pushButton_clicked()
{

ui->pushButton->hide();


    QUrl ur("test");
    startRequest();


}

bool once=1;
void Form::startRequest()
{
    QThread::msleep(1);
    qnam.clearAccessCache();
    qnam.clearConnectionCache();
    
    reply = qnam.get(QNetworkRequest(url));
    connect(reply, &QNetworkReply::finished, this, &Form::httpFinished);

}

void Form::httpFinished()
{
    disconnect(reply, &QNetworkReply::finished, this, &Form::httpFinished);
    reply->deleteLater();

    emit nextReq();
 //  reply=NULL;
}

JonB

@Q139
Not knowing much about electricity, your picture above is how I power my PCs. Is there anything wrong with this setup?

Q139

@JonB
For temporary solution i think its great way to power computer/s if you really needed the power and had no access to better options.

One possible bad scenario is ,if you add more PCs and increase load at some point it may act as fuse, after breaking connection short circuit and consequently breaking main fuse of the house if reasonable short circuit occurs, or just catching fire.

Maybe something corrodes over the years and starts to heat, then you will get FREE house heater out of it as well in addition to powering pc, works best on wooden house.

Connections using proper wires:

Bonnie

I replace you sleep with timer, does this change anything?

#include "form.h"
#include "ui_form.h"
#include <QtNetwork>
#include <QUrl>
#include "QProgressDialog"
#include "QMessageBox"
#include "QTextBlock"
#include "qcoreapplication.h"
Form::Form(QWidget *parent) :
    QWidget(parent),
    ui(new Ui::Form)
{
    ui->setupUi(this);

#ifndef QT_NO_SSL
    connect(&qnam, &QNetworkAccessManager::sslErrors,
            this, &Form::sslErrors);
#endif

}

Form::~Form()
{
    delete ui;
}

void Form::on_pushButton_clicked()
{
    ui->pushButton->hide();
    startRequest();
}

void Form::startRequest()
{    
    qnam.clearAccessCache();
    reply = qnam.get(QNetworkRequest(url));
    connect(reply, &QNetworkReply::finished, this, &Form::httpFinished);
}

void Form::httpFinished()
{
    reply->deleteLater();
    QTimer::singleShot(1000, this, &Form::startRequest);
}

Q139

@Bonnie Unfortunately not.

QNetworkAccessManager qnam;
QNetworkReply *reply;

void Form::startRequest()
{
    qnam.clearAccessCache();
    reply = qnam.get(QNetworkRequest(QUrl("www.microsoft.com")));
    connect(reply, &QNetworkReply::finished, this, &Form::httpFinished);
}

void Form::httpFinished()
{
     disconnect(reply, &QNetworkReply::finished, this, &Form::httpFinished);
    reply->deleteLater();
    QTimer::singleShot(1, this, &Form::startRequest);
}

This sums the core code up..
If someone plans to use it as ddos code it will probably flood your own memory up instead.

Bonnie

@Q139 Actually, I don't get a increasing memory while testing the last and the previous code of yours.
I wonder if it needs to be tested with a url that have a big size of reply data...
I'm using Qt 5.12 though...

Christian Ehrlicher

@Q139 said in How to use QNetworkAccessManager without leaking memory.:

If someone plans to use it as ddos code it will probably flood your own memory up instead.

No it won't - there is no leak in this code anymore.
The disconnect() is not needed though.