qt file uploading, Connection closed error
-
Hi,
I need the application to upload images to our server, it may have 2000-5000 images and each file is about 20-30kb, the application upload one file at a time.
When upload around 30-50 files qt shows an error "Connection closed", I don't know what causing this, my hosting is in Godaddy and I contacted and they there is no problem in the server, my hosting account is a shared server.
If upload again it shows the same error but if uploaded after 20 or 30, I can upload another 10-20 files but it's not a good workaround but now I am using this technique.void myServer::uploadFiles() { //qDebug() << "start uploading files"; sqlQuery.exec("SELECT data_contentType, data_createdDate, data_eventId, data_name, data_type, " "data_teamMemberId, dataId FROM data WHERE uploaded = '0' AND converted = '1' LIMIT 1 "); if (sqlQuery.next()) { qDebug() << "uploading : "+sqlQuery.value(3).toString(); QHttpMultiPart *multiPart = new QHttpMultiPart(QHttpMultiPart::FormDataType); QHttpPart data_contentType; data_contentType.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"data_contentType\"")); data_contentType.setBody(sqlQuery.value(0).toString().toLatin1()); QHttpPart data_createdDate; data_createdDate.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"data_createdDate\"")); data_createdDate.setBody(sqlQuery.value(1).toString().toLatin1()); QHttpPart data_eventId; data_eventId.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"data_eventId\"")); data_eventId.setBody(sqlQuery.value(2).toString().toLatin1()); QHttpPart data_name; data_name.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"data_name\"")); data_name.setBody(sqlQuery.value(3).toString().toLatin1()); QHttpPart data_type; data_type.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"data_type\"")); data_type.setBody(sqlQuery.value(4).toString().toLatin1()); QHttpPart data_teamMemberId; data_teamMemberId.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"data_teamMemberId\"")); data_teamMemberId.setBody(sqlQuery.value(5).toString().toLatin1()); QHttpPart dataId; dataId.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"dataId\"")); dataId.setBody(sqlQuery.value(6).toString().toLatin1()); QHttpPart imagePart; imagePart.setHeader(QNetworkRequest::ContentTypeHeader, QVariant("image/jpeg")); imagePart.setHeader(QNetworkRequest::ContentDispositionHeader, QVariant("form-data; name=\"image\"; filename=\"1.jpg\"")); QFile *file = new QFile(settings.value("app/workingDir").toString()+"w_500/"+sqlQuery.value(3).toString()); file->open(QIODevice::ReadOnly); imagePart.setBodyDevice(file); file->setParent(multiPart); // we cannot delete the file now, so delete it with the multiPart multiPart->append(data_contentType); multiPart->append(data_createdDate); multiPart->append(data_eventId); multiPart->append(data_name); multiPart->append(data_type); multiPart->append(data_teamMemberId); multiPart->append(dataId); multiPart->append(imagePart); QUrl url(settings.value("app/serverUrl").toString()+"update/upload_files"); QNetworkRequest request(url); //QNetworkAccessManager manager; QNetworkAccessManager * manager = new QNetworkAccessManager(); QNetworkReply *reply = manager->post(request, multiPart); multiPart->setParent(reply); // delete the multiPart with the reply // here connect signals etc. connect(manager, SIGNAL(finished(QNetworkReply*)), SLOT(fileUploadFinished(QNetworkReply*))); } } void myServer::fileUploadFinished(QNetworkReply *replay) { // Checking replay if (replay->isReadable()) { // Checking errors if(replay->error() == QNetworkReply::NoError) { QByteArray strreplay = replay->readAll(); // update ststus check if(strreplay != "file uploading failed") { qDebug() << "upload sucessfull "+strreplay; sqlQuery.exec("SELECT data_name FROM data WHERE dataId = '"+strreplay+"'"); if (sqlQuery.next()) { // remove converted image QFile imgfile(settings.value("app/workingDir").toString()+"w_500/"+sqlQuery.value(0).toString()); imgfile.remove(); // update database sqlQuery.exec("UPDATE data SET uploaded = '1' WHERE dataId = '"+strreplay+"' "); } } else { qDebug() << strreplay; } } else { qDebug() << replay->errorString(); QTimer::singleShot(20000, this, SLOT(waitupload())); return; } } replay->close(); this->uploadFiles(); }
Thanks
-
@4j1th
I'm not sure if this is the reason, but you created a lot of objects QNetworkAccessManager and QNetworkReply and never delete them. According to documentation:
"One QNetworkAccessManager should be enough for the whole Qt application."
"Currently, for the HTTP protocol on desktop platforms, 6 requests are executed in parallel for one host/port combination."
May be it is better to make QNetworkAccessManager member of the class myServer and to not create new object in uploadFiles.QNetworkAccessManager *manager; // in header file
manager = new QNetworkAccessManager(); // In the constructor of myServer
myServer::uploadFiles() ... QNetworkReply *reply = manager->post(request, multiPart);
And to delete QNetworkReply objects:
fileUploadFinished(QNetworkReply *replay) ... replay->close(); replay->deleteLater();
And what is doing the slot waitupload()?
-
@Stoyan I think this post solves the new object issue
https://stackoverflow.com/questions/23267925/when-to-delete-qnetworkaccessmanager-pointer
// Delete object whose signal triggered this slot. QObject *manager = sender(); manager->deleteLater();
But this doesn't solve the connection error issue
-
@4j1th
Did you addreplay->deleteLater();
What is the result of replay->errorString(); ?
The code you post should work if you create instance of QNetworkAccessManager every time.
Did you left in code this row:QNetworkAccessManager * manager = new QNetworkAccessManager();
Did you call function uploadFiles() from multiple places?
I suspect that you create multiple connection to server and there is a limitation for such activity. -
@4j1th said in qt file uploading, Connection closed error:
@Stoyan Thanks for the replay
I tried you said but it save same file copy multiple times in server and don't return the send post data in order
I think I understand why this happens. Try to move this row
connect(manager, SIGNAL(finished(QNetworkReply*)), SLOT(fileUploadFinished(QNetworkReply*)));
in the constructor, right after this:
manager = new QNetworkAccessManager();
Else it creates multiple connections for one object.
-
@4j1th
Can you put in the beginning of fileUploadFinished, before checking for errors, this code:QByteArray strreplay = replay->readAll(); qDebug() << "Reply: " << strreplay; qDebug() << "Reply error code: " << replay->error();
Maybe there is some informative response to some of the requests before terminating connection.
This is the list with all error codes: QNetworkReply::NetworkError -
@Stoyan thank you for that debugging idea
"uploading : 2017_06_18_00_00_00_26/raw/8/images/mUarFdVu5ZZmiOqFjLoTz4lXn.JPG" Reply: "27" Reply error code: QNetworkReply::NetworkError(NoError) "upload sucessfull 27" "uploading : 2017_06_18_00_00_00_26/raw/8/images/sj0DMlogCTmdoUZEtBk8pph0R.JPG" Reply: "28" Reply error code: QNetworkReply::NetworkError(NoError) "upload sucessfull 28" "uploading : 2017_06_18_00_00_00_26/raw/8/images/0nMaAxIhlJtNxOPHystIFv2EW.JPG" Reply: "29" Reply error code: QNetworkReply::NetworkError(NoError) "upload sucessfull 29" "uploading : 2017_06_18_00_00_00_26/raw/8/images/0t9VhPJKZjIMppyzZBknOpb4Z.JPG" Reply: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>408 Request Timeout</title>\n</head><body>\n<h1>Request Timeout</h1>\n<p>Server timeout waiting for the HTTP request from the client.</p>\n<p>Additionally, a 408 Request Timeout\nerror was encountered while trying to use an ErrorDocument to handle the request.</p>\n</body></html>\n" Reply error code: QNetworkReply::NetworkError(UnknownContentError) "Error transferring http://client.mydomain.com/index.php/update/upload_files - server replied: Request Timeout"
-
@4j1th
If error "RemoteHostClosedError" follow the previous error "UnknownContentError", probably the reason is the first error.
You may run the program some more times to collect more data and contact again your service provider with this information.
It is possible to exists some limitation on your account. For example: number of transferred files for a period. Some servers after too many requests for a short time, ask for some type additional identification (captcha for example).And one more thing (not directly connected to the main problem) - you should close and delete the instance of "replay" even when an error occurred.