Memory Leak on QNetworkAccessManager
-
Hello, first of all I would like to apologize for my English, I live in Brazil.
I am having problems with possible Memory Leaks in the QNetworkAccessManager class. I did a lot of research and it seems that I was not the only one to notice such a problem.
I do not know if it's the way I'm doing or if it is really a problem in class.
I have created a function to get the HTML code of certain websites, however whenever this function is called, I notice a small increase in consumption in RAM. I've already tried using the deleteLater() function and several other ways, but to no avail. What else has worked so far is manually deleting the memory pointers, but there are still minor leaks.
Here is the function I created:
#include <QObject> #include <QString> #include <QNetworkAccessManager> #include <QNetworkRequest> #include <QNetworkReply> #include <QEventLoop> #include <QTimer> QString getHTML(QString url) { QNetworkAccessManager *manager = new QNetworkAccessManager(); QNetworkRequest request; QNetworkReply *reply; QEventLoop *loop = new QEventLoop(); QTimer *timeout = new QTimer(); timeout->setSingleShot(true); request.setUrl(QUrl(url)); request.setRawHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"); request.setAttribute(QNetworkRequest::FollowRedirectsAttribute, true); QObject::connect(manager, SIGNAL(finished(QNetworkReply*)), loop, SLOT(quit())); QObject::connect(timeout, SIGNAL(timeout()), loop, SLOT(quit())); reply = manager->get(request); timeout->start(10000); loop->exec(); QString html = reply->readAll(); delete reply; delete loop; delete timeout; delete manager; return html; }Follow the pictures during my tests:
• On idle:
• After 1 minute:
• After 6 minutes:
• After 28 minutes:
I will also leave the link of my project as an example of the problem in GitHub. I am using the MingW 32bit compiler with Qt 5.11.1
https://github.com/senjaxus/Memory-Leak-on-QNetworkAccessManagerThanks to everyone who can help find the problem.
QObject::connect(manager, SIGNAL(finished(QNetworkReply*)), loop, SLOT(quit())); QObject::connect(timeout, SIGNAL(timeout()), loop, SLOT(quit())); reply = manager->get(request); timeout->start(10000); loop->exec();Why don't you use Signals and Slots like the where designed? Using local event loops is advanced stuff. I bet the effect you see is because of that.
-
Apart from the really strange code construct - task manager is not a tool to measure the real memory footprint of an application and it really is not a tool to check for memory leaks...
valgrind on linux does not find anything which looks like a memleak here.Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
Visit the Qt Academy at https://academy.qt.io/catalog -
QObject::connect(manager, SIGNAL(finished(QNetworkReply*)), loop, SLOT(quit())); QObject::connect(timeout, SIGNAL(timeout()), loop, SLOT(quit())); reply = manager->get(request); timeout->start(10000); loop->exec();Why don't you use Signals and Slots like the where designed? Using local event loops is advanced stuff. I bet the effect you see is because of that.
Hi @aha_1980 .
I did it this way because the QNetworkAccessManager is asynchronous. I need to get it to wait for finalization or receive a timeout after a certain time, in the example case, 10 seconds.
Could guide me if there's a better way to do this?
-
Apart from the really strange code construct - task manager is not a tool to measure the real memory footprint of an application and it really is not a tool to check for memory leaks...
valgrind on linux does not find anything which looks like a memleak here.Really the Windows task manager is not the best tool to check this problem. But I did not need another tool to identify that there is a leak, if there is one. It is very strange to see the program consuming RAM this way, this in the long run can be disastrous.
-
Hi @aha_1980 .
I did it this way because the QNetworkAccessManager is asynchronous. I need to get it to wait for finalization or receive a timeout after a certain time, in the example case, 10 seconds.
Could guide me if there's a better way to do this?
I'd use a QTimer with 10 seconds timeout. When timer slot is fired, check if the request is finished, otherwise abort it.
Am I missing something?
Qt has to stay free or it will die.
-
I'd use a QTimer with 10 seconds timeout. When timer slot is fired, check if the request is finished, otherwise abort it.
Am I missing something?
True, I had forgotten that part. I'll do this and see if the problem continues.
-
@Maickonn-Richard said in Memory Leak on QNetworkAccessManager:
Really the Windows task manager is not the best tool
It's not even a tool to catch such kind of errors in any way. It may be a hint but nothing more... and since valgrind (which is a tool to catch exactly such kind of problems) does not show anything... but do whatever you like.
Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
Visit the Qt Academy at https://academy.qt.io/catalog -
@Maickonn-Richard said in Memory Leak on QNetworkAccessManager:
Really the Windows task manager is not the best tool
It's not even a tool to catch such kind of errors in any way. It may be a hint but nothing more... and since valgrind (which is a tool to catch exactly such kind of problems) does not show anything... but do whatever you like.
I thank you very much for your help with Valgrind! unfortunately currently I can not access this tool.
-
Hello, first of all I would like to apologize for my English, I live in Brazil.
I am having problems with possible Memory Leaks in the QNetworkAccessManager class. I did a lot of research and it seems that I was not the only one to notice such a problem.
I do not know if it's the way I'm doing or if it is really a problem in class.
I have created a function to get the HTML code of certain websites, however whenever this function is called, I notice a small increase in consumption in RAM. I've already tried using the deleteLater() function and several other ways, but to no avail. What else has worked so far is manually deleting the memory pointers, but there are still minor leaks.
Here is the function I created:
#include <QObject> #include <QString> #include <QNetworkAccessManager> #include <QNetworkRequest> #include <QNetworkReply> #include <QEventLoop> #include <QTimer> QString getHTML(QString url) { QNetworkAccessManager *manager = new QNetworkAccessManager(); QNetworkRequest request; QNetworkReply *reply; QEventLoop *loop = new QEventLoop(); QTimer *timeout = new QTimer(); timeout->setSingleShot(true); request.setUrl(QUrl(url)); request.setRawHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"); request.setAttribute(QNetworkRequest::FollowRedirectsAttribute, true); QObject::connect(manager, SIGNAL(finished(QNetworkReply*)), loop, SLOT(quit())); QObject::connect(timeout, SIGNAL(timeout()), loop, SLOT(quit())); reply = manager->get(request); timeout->start(10000); loop->exec(); QString html = reply->readAll(); delete reply; delete loop; delete timeout; delete manager; return html; }Follow the pictures during my tests:
• On idle:
• After 1 minute:
• After 6 minutes:
• After 28 minutes:
I will also leave the link of my project as an example of the problem in GitHub. I am using the MingW 32bit compiler with Qt 5.11.1
https://github.com/senjaxus/Memory-Leak-on-QNetworkAccessManagerThanks to everyone who can help find the problem.
If I may,
this is one way, how you can make use of the assynchon function:
QString getHTML(QString url) { QNetworkRequest request; request.setUrl(QUrl(url)); request.setRawHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"); request.setAttribute(QNetworkRequest::FollowRedirectsAttribute, true); QNetworkAccessManager *manager = new QNetworkAccessManager(this); connect(manager, &QNetworkAccessManager::finished, this, [=](QNetworkReply *reply){ if(reply.error() == QNetworkReply::NoError) emit htmlRead(QString(reply->readAll()); else emit htmlError(reply->error()); manager->deleteLater(); reply->deleteLater(); }); manager->get(request); QTimer *timeout = new QTimer(this); timeout->start(10000); connect(timeout &QTimer::timeout, manager, [=]{ emit htmlTimeout(); manager->deleteLater(); timeout->deleteLater(); }); connect(manager, &QNetworkAccessManager::finished, timeout, QTimer::deleteLater); }You'll have to create slots that connect to the signals
void htmlTimeout(),void htmlError(QNetworkReply::NetworkError)andvoid htmlRead(QString)and proceed from there.This should work, but it's untested.
Be aware of the Qt Code of Conduct, when posting : https://forum.qt.io/topic/113070/qt-code-of-conduct
Q: What's that?
A: It's blue light.
Q: What does it do?
A: It turns blue. -
If I may,
this is one way, how you can make use of the assynchon function:
QString getHTML(QString url) { QNetworkRequest request; request.setUrl(QUrl(url)); request.setRawHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"); request.setAttribute(QNetworkRequest::FollowRedirectsAttribute, true); QNetworkAccessManager *manager = new QNetworkAccessManager(this); connect(manager, &QNetworkAccessManager::finished, this, [=](QNetworkReply *reply){ if(reply.error() == QNetworkReply::NoError) emit htmlRead(QString(reply->readAll()); else emit htmlError(reply->error()); manager->deleteLater(); reply->deleteLater(); }); manager->get(request); QTimer *timeout = new QTimer(this); timeout->start(10000); connect(timeout &QTimer::timeout, manager, [=]{ emit htmlTimeout(); manager->deleteLater(); timeout->deleteLater(); }); connect(manager, &QNetworkAccessManager::finished, timeout, QTimer::deleteLater); }You'll have to create slots that connect to the signals
void htmlTimeout(),void htmlError(QNetworkReply::NetworkError)andvoid htmlRead(QString)and proceed from there.This should work, but it's untested.
@J.Hilk very nice example!
And it perfectly shows how well signals works together with lambdas...
Qt has to stay free or it will die.
-
@J.Hilk very nice example!
And it perfectly shows how well signals works together with lambdas...
-
Well guys, after leaving the software running for a long period with the code I posted, I noticed that the consumption of RAM stabilizes. That is, leaks are not happening.
I thank all who cooperated, the help of you all were immense, you gave me a number of alternative ways in which I had not considered before.
A big hug to everyone!
-
@J.Hilk very nice example!
And it perfectly shows how well signals works together with lambdas...
