Post upload file send only 16kb

vicsoftware

Hi,

QTWindow constructor:


m_netManager = new QNetworkAccessManager{ this };
connect( m_netManager, & QNetworkAccessManager::finished, this, & QTWindow::onRequestFinished );

void QTWindow::sendImageHttp( const QString & jpgPath )
{
log( "Upload..." );

QUrl url{ "https://xxxxxxxxxxxx/photo" };

QNetworkRequest request{ url };

m_requestFile = { new QFile{ jpgPath } };
if( not m_requestFile->open( QIODevice::ReadOnly ) )
{
    log( "Failed to load file: " + jpgPath );
    return;
}
const auto fileSize{ m_requestFile->size() };
QByteArray fileLength;
fileLength.setNum( fileSize );
QFileInfo fi{ jpgPath };

request.setRawHeader( "User-Agent", "SlabTools/1.0" );
request.setRawHeader( "Accept-Encoding", "gzip, deflate" );
request.setRawHeader( "Accept", "*/*" );
request.setRawHeader( "Connection", "keep-alive" );
request.setRawHeader( "authorization", "Basic xxxxxxxxxxxxxx" );
request.setRawHeader( "cache-control", "no-cache" );
request.setRawHeader( "slb", "00199975" );
request.setRawHeader( "spytnik", "0" );
request.setRawHeader( "Content-Type", "image/jpeg" );
request.setRawHeader( "Content-Length", fileLength );

m_requestReply = { m_netManager->post( request, m_requestFile ) };
connect( m_requestReply, & QNetworkReply::uploadProgress, this, & QTWindow::onReplyUploadProgress );
connect( m_requestReply, & QNetworkReply::finished, this, & QTWindow::onRequestFinished );
connect( m_requestReply, QOverload< QNetworkReply::NetworkError >::of( & QNetworkReply::error ),
         [ = ]( QNetworkReply::NetworkError code )
{
    log( "Upload error code: " + QString::number( code ) );
    log( "Upload error: " + m_requestReply->errorString() );
});

);
log( "Upload success." );
}


Upload < 16kb file - success.

If file > 16kb then after several attempts ( wait about 1 minute for every attempt ) and server close connection with error 2.

Upload...
Upload success.
16384 27025
16384 27025
16384 27025
16384 27025
Upload error code: 2
0 0
Upload finished.
Upload error: Connection closed



Here is python code and it works with any file:

try:
headers = {
'authorization': "Basic xxxxxxxxxxxxxxxx",
'cache-control': "no-cache",
"slb": "00199975",
"spytnik": "0"
}
file = open("test2.jpg", "rb")

response = requests.request("POST", verify=True, url="https://xxxxxxxxxxxxxx/photo", data=file, headers=head>

print( "Response:", response.json() )

except Exception as e:
print( "Exception:", e )



QHttpMultipart same issue.

It is Ubuntu 20, QT5.15.2. Also i tried on Windows QT 5.15 same issue.

Please help.

Best regards, Viktor.

JonB

@vicsoftware
If I'm not mistaken: all/most of you variables are local to your method, they will go out of scope when then exits, which is more or less as soon as you as your do the post(). You need to keep necessaries in existence till the request has finished.

Python code is more likely to keep objects around than C++ code, where you have to be explicit.

Let us know if that is indeed the reason here?

EDIT Upon reflection, I'm not so sure. I now see you are using m_... variables.

vicsoftware

@JonB Really file is class member variable ( fixed ), m_netManager, m_replyRequest too.
Should i QNetworkRequest make global too?

jsulm

@vicsoftware said in Post upload file send only 16kb:

Should i QNetworkRequest make global too?

Not global but member variable.

JonB

@vicsoftware
I am unsure whether a QNetworkRequest instance still needs to live after you have done m_requestReply = { m_netManager->post( request, m_requestFile ) };, or at that point only the reply object is needed.

Why don't you shove in a static or a new temporarily and see if that makes any difference? [EDIT @jsulm has seemed to say you do need this to persist, so as he says it will want to be a member variable.]

I assume your QTWindow persists? And you do not dispose m_requestFile = { new QFile{ jpgPath } };?

And just BTW: do you eventually dispose m_requestFile and close it first?

vicsoftware

@JonB When i should close file? After post statement?

QTWindow is simple QMainWindow class.

Updated

m_requestReply = { m_netManager->post( request, m_requestFile ) };
m_requestFile->close();

Upload...
Upload success.
QIODevice::read (QFile, "/home/vsw/test2.jpg"): device not open
Upload error code: 99
0 0
Upload finished.
Upload error: Unknown error

JonB

@vicsoftware said in Post upload file send only 16kb:

@JonB When i should close file? After post statement?

data must be open for reading and must remain valid until the finished() signal is emitted for this reply.

So in onRequestFinished you need to close it, and (presumably) delete m_requestFile. You also need that done in the error case (assuming that does not generate a QNetworkReply::finished).

In that light, your placement of m_requestFile->close(); above is wrong now and will cause the device not open, get that fixed and try again.

vicsoftware

@JonB Tried close file and close reply:

void QTWindow::onRequestFinished( QNetworkReply * reply )
{
    reply->close();
    reply->deleteLater();
//    m_requestFile->close();
//    delete m_requestFile;
    log( "Upload finished." );
}

Same issue.

As i see finished does not fired. It waits some responce or somthing else.

Upload...
Upload success.
16384 27025
16384 27025

JonB

@vicsoftware
To be clear: I only put those remarks in for your good coding behaviour (hence the "And just BTW:"). They were not expected to have any effect on whatever your upload problem is.

vicsoftware

@JonB Yes, i simply try all advises, cause i fight with this second day. Read all forums and decided to write here.
I can take any other c++ library, but should understand it.

First i tried Multipart https://doc.qt.io/qt-5/qhttpmultipart.html
Issue exactly the same.
But python and curl works on my pc and other person.

JonB

@vicsoftware
I haven't Googled, but if I were in your position I would go find the simplest example of a file upload example out there for Qt. There must be many, maybe even on the docs page, there are also probably answers with it on this forum. Try the standard code others use and see if that works on bigger file.

SGaist

Hi,

Did you also check that your headers are all correct ?

fcarney

Check for limits on the server you are posting to. Does other software allow you to post bigger files? We have a server we wrote in python that had a limit of 40MB. It was a self imposed limit.

vicsoftware

@fcarney Any files with any libraries. But not Qt. May be it is bug?

fcarney

Looks like some sort of config limit:
https://doc.qt.io/qt-5/qnetworkrequest.html#http2Configuration

vicsoftware

@SGaist I think yes. I tried many combinations. I compare headers from curl and python requests and they works. May be Qt network engine needs additional headers, but it is abnormal.

Now it works with curl cpp wrapper in same class in same function without any problems.

SGaist

Did you try following the example of QHttpMultiPart ? Adding only the minimal set of headers like the authentication and nothing else ?

vicsoftware

@fcarney Tried:

QHttp2Configuration conf = request.http2Configuration();
conf.setMaxFrameSize( 100 * 1024 * 1024 );
request.setHttp2Configuration( conf );

And got this:

Upload...
qt.network.http2: Maximum frame size to advertise is invalid
Upload success.
16384 27025
16384 27025
27025 27025
0 0
Upload finished.

10 Mb file

Upload...
qt.network.http2: Maximum frame size to advertise is invalid
Upload success.
16384 10669810
16384 10669810
16384 10669810
16384 10669810
Upload error code: 2
0 0
Upload finished.
Upload error: Connection closed

It is something new.

And it is also very slow.

vicsoftware

@SGaist Yes, i did all Googled combinations to upload file via QT nm.post, including qtmultipart examples from stackoverflow. Always failed.

fcarney

I just looked at some code we use. It uses multipart and we can send files larger than 16KB. I cannot share it though.