QNetworkAccessManager no longer POSTing content

sipickles

Hi,

After upgrading from Qt 4.7.2 to 4.8.5, one of my unit tests started failing.

In the test I start a QTcpServer then do a POST to it with QNetworkAccessManager. The server gets the POST request but there is no longer any content. It worked with 4.7.2.

Strangely the content-length header is correctly marked. Here's the request as received by the server:
@POST /load HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Content-Length: 9
Connection: Keep-Alive
Accept-Encoding: gzip
Accept-Language: en-US,*
User-Agent: Mozilla/5.0
Host: 127.0.0.1:50000

@
There are actually 2 blanks lines logged after the 'Host' line. I suspect this is related. As I understand it, the header and content are separated by \r\n.

As expected the content ("TEST_DATA") is 9 bytes long. I've tried lots of different Content-Type settings but no difference.

Here is the test which makes the POST:
@std::string gCommandResult;
std::string gArgResult;

void requestHandler(std::string command, std::string arg)
{
gCommandResult = command;
gArgResult = arg;
}

void MyServerTest::testSetRequestHandler()
{
const bool success = _server->start(kTestHost, kTestPort, kTestPortRange);
QVERIFY2(success, "Server should return success");

_server->setRequestHandler(requestHandler);
QVERIFY(gCommandResult.empty());
QVERIFY(gArgResult.empty());

// Make a POST request
const std::string commandExpected = "load";
const std::string argExpected = "TEST_DATA";

std::stringstream s;
s << "http://" << kTestHost << ":" << kTestPort << "/" << commandExpected;
QUrl url(s.str().c_str());
QNetworkRequest request(url);
request.setHeader(QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded");
QByteArray data(argExpected.c_str());

// This is async
QNetworkReply* asyncReply = _client->post(request, data); // alloc
const bool noTimeOut = UITestUtils::waitForSignal(asyncReply, SIGNAL(finished()), 1000);

// Tidy up before test checks
asyncReply->disconnect();
asyncReply->deleteLater();
_server->close();

QVERIFY2(noTimeOut, "The post request timed out");
QCOMPARE(gCommandResult, commandExpected);
std::cout << gArgResult << "," << argExpected << std::endl;
QCOMPARE(gArgResult, argExpected);
}
@

And here is the server code, connected to the readyRead signal:
@void MyServer::handleRequest()
{
QTcpSocket* client = (QTcpSocket*)sender();
if (!client || client->state() != QAbstractSocket::ConnectedState)
return;

QByteArray data = client->readAll();
std::string request(data.constData());
qint64 bytesRead = request.size();

if (gHelpDebugVerbose) {
std::ostringstream ss;
ss << std::endl << "handleRequest:::::::::::::::::::::::::::::: bytes: " << bytesRead << std::endl << request << std::endl;
log(ss);
}

const std::string requestType = request.substr(0,4);
if (requestType == "HEAD") {
// HEAD request is like a GET, but wants headers only. Might come through on linux before the GET
return handleRequestHEADOrGET(client, request, true);
}
else if (requestType == "GET ") {
return handleRequestHEADOrGET(client, request, false);
}
else if (requestType == "POST") {
return handleRequestPOST(client, request);
}
else {
if (gHelpDebugVerbose) {
std::ostringstream ss;
ss << std::endl << "handleRequest error detecting request type" << std::endl;
log(ss);
}
return; // process no more
}
}

void MyServer::handleRequestPOST(QTcpSocket* client, const std::string& request)
{
Log("handleRequestPOST" << std::endl);

std::string command, arg;
if (!parseRequestPOST(request, command, arg)) {
sendPageNotFound(client, "");
return;
}

if (gHelpDebugVerbose) {
std::ostringstream ss;
ss << "handleRequestPOST - Command " << command << " - Arg " << arg << std::endl;
log(ss);
}

if (_handlerCallback) {
Log("handleRequestPOST passing to handlerCallback " << _handlerCallback << std::endl);
_handlerCallback(command, arg);
}
else {
Log("handleRequestPOST - no callback assigned" << std::endl);
}

// Send an OK
const std::string response = "HTTP/1.1 200 OK\r\n\r\n";
client->write(response.c_str());

// Disconnecting shouldn't be required but seems to be.
// Without it, the write seems buffered, perhaps by Mac OSX.
client->disconnectFromHost();

}

@

If anyone knows what has changed in 4.8 to cause this, I'd be grateful. Thanks

sipickles

I've revisited this bug and used Wireshark to analyse the network traffic.

QNetworkAccessManager is NOT posting the content! Here's the wireshark capture.

@....E....4@.@...
.........0.Po^9.
M<.....~........
..Y...Y.POST /lo
ad HTTP/1.1..Con
tent-Type: text/
plain..Content-L
ength: 9..Connec
tion: Keep-Alive
..Accept-Encodin
g: gzip..Accept-
Language: en-GB,
*..User-Agent: M
ozilla/5.0..Host
: 127.0.0.1:5000
0....@

The last two lines in hex:
@3a 20 31 32 37 2e 30 2e 30 2e 31 3a 35 30 30 30
30 0d 0a 0d 0a@

As you can see the last line contains the final zero of '50000' and the four dots correspond the \r\n\r\n. There is no content. Bah!

I've debugged into the Qt source as far as possible and it looks like the QNetworkReply is created, loaded with the request content and added to the Qt event queue correctly.

Can anyone confirm they can POST data in 4.8.5? Thanks

.... and Merry Xmas!

SGaist

Hi,

Can you share a minimal compilable example that reproduce the behavior ?

sipickles

I've stripped it down to the simplest possible app and I dont get the problem. The POST data is commumicated correctly.

However, I've also noticed that I get two readyRead signals in my broken test case. In each case, readAll only gets the header, no content.

I'll try to get better repro.

sipickles

Sorry, I made a mistake. In my simplest case I was still linking to 4.7.2, hence no problem.

Now I've linked to 4.8.5, the problem reappears. Here's the code:

@// HelpServer.h //////////////////////////////////////////////////////////////
#include <sstream>
#include <iostream>
#include <QtNetwork/QTcpSocket>
#include <QtNetwork/QTcpServer>
#include <QtNetwork/QNetworkAccessManager>
#include <QtNetwork/QNetworkRequest>
#include <QtNetwork/QNetworkReply>
#include <QtCore/QObject>
#include <QtCore/QUrl>
#include <QtCore/QBuffer>
#include <QtCore/QTimer>
#include <QtCore/QCoreApplication>

const std::string kHost = "127.0.0.1";
const int kPort = 5000;

class HelpServer : public QTcpServer
{
Q_OBJECT

protected slots:
void handleRequest()
{
QTcpSocket* client = (QTcpSocket*)sender();
if (!client || client->state() != QAbstractSocket::ConnectedState)
return;

QByteArray data = client->readAll();
std::string request(data.constData());

std::ostringstream ss;
ss << std::endl << "handleRequest" << std::endl << request << std::endl;
std::cout << ss.str() << std::endl;

// Send an OK
const std::string response = "HTTP/1.1 200 OK\r\n\r\n";
client->write(response.c_str());
client->disconnectFromHost();

}

void incomingConnection(int socketfd)
{
QTcpSocket* client = new QTcpSocket(this); // Server is parent so owns this heap memory
client->setSocketDescriptor(socketfd);
connect(client, SIGNAL(readyRead()), this, SLOT(handleRequest()));
}

public:
bool start()
{
QHostAddress hostAddress(kHost.c_str());
quint16 p(kPort);

if (!listen(hostAddress, p)) {
  std::cout << "Listen failed" << std::endl;
  return false;
}

return true;

}
};

// main.cpp //////////////////////////////////////////////////////////////
#include <sstream>
#include <iostream>
#include <QtNetwork/QTcpSocket>
#include <QtNetwork/QTcpServer>
#include <QtNetwork/QNetworkAccessManager>
#include <QtNetwork/QNetworkRequest>
#include <QtNetwork/QNetworkReply>
#include <QtCore/QObject>
#include <QtCore/QUrl>
#include <QtCore/QBuffer>
#include <QtCore/QTimer>
#include <QtCore/QCoreApplication>

#include "HelpServer.h"

// Provide a temporary event loop while we wait for an async network operation
bool waitForSignal(QObject *sender, const char *signal, int timeout)
{
QEventLoop loop;
QTimer timer;
timer.setInterval(timeout);
timer.setSingleShot(true);

loop.connect(sender, signal, SLOT(quit()));
loop.connect(&timer, SIGNAL(timeout()), SLOT(quit()));
timer.start();
loop.exec();

return timer.isActive();
}

int main(int argc, char *argv[])
{
QCoreApplication app(argc, argv);
std::cout << QT_VERSION_STR << std::endl;

HelpServer server;
QNetworkAccessManager client;

server.start();

std::string postData("HelloWorld");

std::stringstream s;
s << "http://" << kHost << ":" << kPort;
QUrl url(s.str().c_str());
QNetworkRequest request(url);
request.setHeader(QNetworkRequest::ContentTypeHeader, "text/plain");
QByteArray data(postData.c_str());

std::cout << "Posting: " << postData << std::endl;
// This is async
QNetworkReply* asyncReply = client.post(request, data); // alloc
const bool noTimeOut = waitForSignal(&client, SIGNAL(finished(QNetworkReply*)), 1000);

asyncReply->disconnect();
asyncReply->deleteLater();

return 0;
}@

I used moc on HelpServer.h, then built it with:

g++ -v -arch x86_64 -o postTest -DQT_BUILD_CORE_LIB -F/workspace/pickles/Thirdparty/Qt/4.8.5/bin/mac-64-x86-release-10.6/frameworks -framework QtCore -framework QtNetwork -I/workspace/pickles/Thirdparty/Qt/4.8.5/src/include/ postTest.cpp moc-HelpServer.cpp

sipickles

Still sufffering with this bug :(

Code_ReaQtor

can you try to change this line:
@ request.setHeader(QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded");@

into this:
@request.setHeader(QNetworkRequest::ContentTypeHeader,
"application/octet-stream");@

and see if the behaviour changes. If I am not mistaken, you can't POST with "application/x-www-form-urlencoded" (i just observed this behaviour in jQuery).

sipickles

Thanks for the advice, I tried octet-stream but no change

nib952051

you should try increase timeout for fast look;
another advice is to change return 0 to return app.exec() to start event loop

stop application you can by connecting reply signal finished to app slot quit