Unsolved QNetworkReply and setReadBufferSize problem
-
@VRonin last one before i stop flood this topic : so i don't need to do this when i call a method on member variable right ?
... private: SFtpClient m_sftp;// ... QObject::connect(connect_btn,&QPushButton::clicked,[=](){ m_sftp->connectToMachine("user@10.81.100.100"); });
or
private: SFtpClient m_sftp;// ... QObject::connect(connect_btn,&QPushButton::clicked,m_sftp,[=](){ m_sftp->connectToMachine("user@10.81.100.100"); });
-
@LeLev said in QNetworkReply and setReadBufferSize problem:
so i don't need to do this when i call a method on member variable right ?
Depends. If the signal emitter can't survive
this
(e.g.connect_btn
is a child ofthis
) then no, but if there's a chanceconnect_btn
can still emit a signal oncethis
was destroyed (and hencem_sftp
was destroyed as well) then you have to addthis
or&m_sftp
as 3rd argument toconnect
to make it safe -
QSaveFile is part of Qt core, it will work on all platforms
The problem is the size of file, not the compatibility.
I understand something's broken at Qt so it won't work. I can't find it in QTBUG. There are traces in version 4.8, but it says "Solved".
Perhaps a "bug request" should be opened... -
@Hergest said in QNetworkReply and setReadBufferSize problem:
Perhaps a "bug request" should be opened
You can do it
-
@Hergest said in QNetworkReply and setReadBufferSize problem:
Can anyone confirm to me that it doesn't work?
It works just fine, that is it sets the read buffer size, which has nothing to do with your problem. As @VRonin said: open a file and stream the network data into it. Whenever you have an opening in the event loop, e.g. you can have a timer or connect to the
bytesWritten
from theQIODevice
, read from the file and process its contents.I understand something's broken at Qt so it won't work.
What won't work? If you mean
setReadBufferSize
, then you're wrong, it works, just it does not do what you expect it to do.@LeLev said in QNetworkReply and setReadBufferSize problem:
last one before i stop flood this topic : so i don't need to do this when i call a method on member variable right ?
You don't need to as long as you understand that the sender is going to be used as context if you don't specify anything. A rule of thumb to avoid problems that are hard to diagnose is just to always provide the context explicitly.
-
@kshegunov said in QNetworkReply and setReadBufferSize problem:
What won't work? If you mean setReadBufferSize, then you're wrong, it works, just it does not do what you expect it to do.
Looking the help:
Sets the size of the read buffer to be size bytes. The read buffer is the buffer that holds data that is being downloaded off the network, before it is read with QIODevice::read(). Setting the buffer size to 0 will make the buffer unlimited in size.QNetworkReply will try to stop reading from the network once this buffer is full (i.e., bytesAvailable() returns size or more), thus causing the download to throttle down as well. If the buffer is not limited in size, QNetworkReply will try to download as fast as possible from the network.
What I expect is that it doesn't keep downloading data until the memory runs out, but that it stops until the buffer is emptied with read.
-
I'm pretty sure readBufferSize() is working as expected since it's an easy implementation: https://code.woboq.org/qt5/qtbase/src/network/access/qnetworkreplyimpl.cpp.html#181 (see nextDownstreamBlockSize() so you must do something wrong elsewhere but without code we can't do anything more.
-
@Hergest said in QNetworkReply and setReadBufferSize problem:
What I expect is that it doesn't keep downloading data until the memory runs out, but that it stops until the buffer is emptied with read.
It can try to throttle the download down if the server supports it, if it doesn't I don't see how this can happen. Consider the simplest scenario where the server just dumps the data over the network (which is what I imagine is happening) the networking stack has a limited buffer and (if set) Qt has a limited buffering, if you don't read the buffer fast enough then it is going to overflow. How do you handle the case where the protocol (or server) has no notion of speed? You can't just leave data pending in the buffers for when you're ready to process it.
-
This is a pseudocode of the function:
... netReply = m_pNetManager->post(req, qb_json); netReply->setReadBufferSize (64*1024); ... /* Reading some head data from netReply*/ ... size_blob = we get the size of the Blob to download from server ... /* Blob Reading from netReply // Loop while (...some conditions...){ ... Debug () << "bytesAvailable=" << netReply->bytesAvailable() << "bufferSize=" << netReply->readBufferSize(); qint64 bytes_read = 0; qint64 bytes_to_read = 1024*64; // chunk to download if ((bytes_read + bytes_to_read) > size_blob) bytes_to_read= size_blob - bytes_read ; qint64 parcial = netReply->read (reinterpret_cast<char*>(buffer_temporal), bytes_to_read); if (parcial > 0){ tmp_file.write (reinterpret_cast<const char*>(buffer_temporal), parcial); bytes_read += parcial; if (bytes_read >= size_blob){ // END READING status = 2; } } ... }
The real code reads the data sent by the server, first a header and then a blob (in 64KB chuncks), then another header and its blob, and so on.
In my tests, the value of "bytesAvailable" increases and surpasses "bufferSize", until the memory collapses.
When I'm debugging it, in a breakpoint, I check that the server stops sending data. As soon as I resume it, "bytesAvailable" continues to increase, and the server continues sending data (data from other blobs).
I have tried many combinations of chunks, ReadBufferSize values, ...This leads me to think...
- The server sends data whenever the client requests it. In the breakpoint it stopped doing it.
- Client keeps asking for data even though "readBuffer" fills up.
-
In the end I had to rewrite the code using QTcpSocket (which does obey the setReadBufferSize limitation), simulating POST requests.