Prevent multiple TCP segments in post request

[vicsoftware]

Hi,

I tried post request to server.

curl and chrome post extension works fine:

curl -X POST -d "\"00000001\"=" -H "Content-Length: 10" http://192.168.0.200/od/2400/01 -v

Frame 524: 222 bytes on wire (1776 bits), 222 bytes captured (1776 bits) on interface 0
Ethernet II, Src: 48:b0:2d:05:a8:be (48:b0:2d:05:a8:be), Dst: NanotecE_00:43:16 (44:aa:e8:00:43:16)
Internet Protocol Version 4, Src: 192.168.0.110, Dst: 192.168.0.200
Transmission Control Protocol, Src Port: 37720, Dst Port: 80, Seq: 1, Ack: 1, Len: 168
    Source Port: 37720
    Destination Port: 80
    [Stream index: 16]
    [TCP Segment Len: 168]
    Sequence number: 1    (relative sequence number)
    [Next sequence number: 169    (relative sequence number)]
    Acknowledgment number: 1    (relative ack number)
    0101 .... = Header Length: 20 bytes (5)
    Flags: 0x018 (PSH, ACK)
    Window size value: 29200
    [Calculated window size: 29200]
    [Window size scaling factor: -2 (no window scaling used)]
    Checksum: 0x8349 [unverified]
    [Checksum Status: Unverified]
    Urgent pointer: 0
    [SEQ/ACK analysis]
    [Timestamps]
    TCP payload (168 bytes)
    TCP segment data (1 byte)
Hypertext Transfer Protocol
    POST /od/2400/01 HTTP/1.1\r\n
    Host: 192.168.0.200\r\n
    User-Agent: curl/7.58.0\r\n
    Accept: */*\r\n
    Content-Length: 10\r\n
    Content-Type: application/x-www-form-urlencoded\r\n
    \r\n
    [Full request URI: http://192.168.0.200/od/2400/01]
    [HTTP request 1/1]
    [Response in frame: 525]
    File Data: 10 bytes
HTML Form URL Encoded: application/x-www-form-urlencoded
    Form item: ""00000001"" = ""
        Key: "00000001"
        Value:

But QT code does not work (Request error: "Connection closed"):

QString url{ "http://" + QString::fromStdString( m_addr ) + "/od/2400/01" };
QNetworkRequest request{ QUrl( url ) };
request.setHeader( QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded" );
request.setHeader( QNetworkRequest::ContentLengthHeader, 10 );
QString data{ "\"00000002\"" };
const auto reply{ m_networkMan->post( request, data.toUtf8() ) };

Frame 422: 64 bytes on wire (512 bits), 64 bytes captured (512 bits) on interface 0
Ethernet II, Src: 48:b0:2d:05:a8:be (48:b0:2d:05:a8:be), Dst: NanotecE_00:43:16 (44:aa:e8:00:43:16)
Internet Protocol Version 4, Src: 192.168.0.110, Dst: 192.168.0.200
Transmission Control Protocol, Src Port: 37718, Dst Port: 80, Seq: 227, Ack: 1, Len: 10
    Source Port: 37718
    Destination Port: 80
    [Stream index: 15]
    [TCP Segment Len: 10]
    Sequence number: 227    (relative sequence number)
    [Next sequence number: 237    (relative sequence number)]
    Acknowledgment number: 1    (relative ack number)
    0101 .... = Header Length: 20 bytes (5)
    Flags: 0x018 (PSH, ACK)
    Window size value: 29200
    [Calculated window size: 29200]
    [Window size scaling factor: -2 (no window scaling used)]
    Checksum: 0x82ab [unverified]
    [Checksum Status: Unverified]
    Urgent pointer: 0
    [SEQ/ACK analysis]
    [Timestamps]
    TCP payload (10 bytes)
    TCP segment data (10 bytes)
[2 Reassembled TCP Segments (236 bytes): #420(226), #422(10)]
    [Frame: 420, payload: 0-225 (226 bytes)]
    [Frame: 422, payload: 226-235 (10 bytes)]
    [Segment count: 2]
    [Reassembled TCP length: 236]
    [Reassembled TCP Data: 504f5354202f6f642f323430302f303120485454502f312e...]
Hypertext Transfer Protocol
    POST /od/2400/01 HTTP/1.1\r\n
    Content-Type: application/x-www-form-urlencoded\r\n
    Content-Length: 10\r\n
    Connection: Keep-Alive\r\n
    Accept-Encoding: gzip, deflate\r\n
    Accept-Language: en-US,*\r\n
    User-Agent: Mozilla/5.0\r\n
    Host: 192.168.0.200\r\n
    \r\n
    [Full request URI: http://192.168.0.200/od/2400/01]
    [HTTP request 1/1]
    File Data: 10 bytes
HTML Form URL Encoded: application/x-www-form-urlencoded
    Form item: ""00000001"" = ""
        Key: "00000001"
        Value:

What i see is 2 tcp segments: headers and data.
How to prevent tcp segments splitting?

Best regards, Viktor.

[Christian Ehrlicher]

@vicsoftware said in Prevent multiple TCP segments in post request:

How to prevent tcp segments splitting?

You can't (unless you change the official protocol - maybe ask at ietf.org if they change it for you :) )

I don't see why it should not work, you code does not match the wireshark dump (0000001 <-> 0000002)

Maybe it has something against the user agent or whatever but the http packet has at least the same data so it can't be the reason.
How do you receive the error?

[vicsoftware]

@Christian-Ehrlicher TCP segments it is my assumption. It is differents what i see. Code value does not matter. Can be "0000000X".
Error received after couple seconds, and wireshark shows that QT sent 2 similar post requests. May be it trying to reconnect.

QT GET request works fine.

Here is how i catch errors:

m_networkMan = new QNetworkAccessManager();
QObject::connect( m_networkMan, & QNetworkAccessManager::finished, this, [ = ]( QNetworkReply * reply )
   {
       if( reply->error() )
       {
           const auto error{ reply->errorString() };
           qDebug() << "Request error: " << error;            
           m_isConnected = false;
           return;
       }
       m_isConnected = true;
       const auto answer{ reply->readAll() };
       qDebug() << "Request answer: " << answer;       
   });

Here is chrome extension output, it works like curl too:

Frame 53: 185 bytes on wire (1480 bits), 185 bytes captured (1480 bits) on interface 0
Ethernet II, Src: 48:b0:2d:05:a8:be (48:b0:2d:05:a8:be), Dst: NanotecE_00:43:16 (44:aa:e8:00:43:16)
Internet Protocol Version 4, Src: 192.168.0.110, Dst: 192.168.0.200
Transmission Control Protocol, Src Port: 39152, Dst Port: 80, Seq: 1, Ack: 1, Len: 131
    Source Port: 39152
    Destination Port: 80
    [Stream index: 7]
    [TCP Segment Len: 131]
    Sequence number: 1    (relative sequence number)
    [Next sequence number: 132    (relative sequence number)]
    Acknowledgment number: 1    (relative ack number)
    0101 .... = Header Length: 20 bytes (5)
    Flags: 0x018 (PSH, ACK)
    Window size value: 29200
    [Calculated window size: 29200]
    [Window size scaling factor: -2 (no window scaling used)]
    Checksum: 0x8324 [unverified]
    [Checksum Status: Unverified]
    Urgent pointer: 0
    [SEQ/ACK analysis]
    [Timestamps]
    TCP payload (131 bytes)
    TCP segment data (2 bytes)
Hypertext Transfer Protocol
    POST /od/2400/01 HTTP/1.1\r\n
    HOST: 192.168.0.200\r\n
    content-type: application/x-www-form-urlencoded\r\n
    content-length: 10\r\n
    \r\n
    [Full request URI: http://192.168.0.200/od/2400/01]
    [HTTP request 1/1]
    [Response in frame: 54]
    File Data: 10 bytes
HTML Form URL Encoded: application/x-www-form-urlencoded
    Form item: ""00000002"" = ""
        Key: "00000002"
        Value:

Also if i using that:

QUrlQuery postData;
postData.addQueryItem( "\"00000002\"", "" );
const auto reply{ m_networkMan->post( request, postData.query().toUtf8() ) };

then QT sent only ' "0000000 ', so i am using inline QString data{ "\"00000002\"" };

It is a bug?

Server API waits Content-Length 10 bytes, Content-Type x-www-form-urlencoded and data "0000000X"=

curl and chrome post works.

[Christian Ehrlicher]

As said - it depends on the OS or whatever how much tcp packets are sent for one request. It does not affect anything on the http protocol. The problem must be somewhere else.

[vicsoftware]

It is Ubuntu 18, Nvidia Jetson AGX, Arm64

Requests executes on same system, why curl and chrome sends single valid post request, but QT sends splitted segments and 2 requests?

[Christian Ehrlicher]

@vicsoftware said in Prevent multiple TCP segments in post request:

but QT sends splitted segments and 2 requests?

It does not send two requests. It sends one request with two tcp packets. Maybe it's how write() is called on the socket or the moon phase or whatever.

[vicsoftware]

@Christian-Ehrlicher No. QT sends 2 request until "closed connection" error and every request has 2 TCP segments.
Curl and chrome sends single request with 1 tcp segment.

Here is 2 first QT request, third is chrome request forth is curl.

As we see length is 64 bytes for QT, and 185 for chrome and 222 for curl.

But QT post has more headers and it shoud be longer bytes. It shows 64 bytes but:

Really tcp has 235 bytes.

[Christian Ehrlicher]

No. QT sends 2 request

This is a new information.

Qt only send two requests when you programmed it to do so.

[vicsoftware]

@Christian-Ehrlicher No. I execute only single post from code. QT trying send second post before closing connection. It's look like.

If it is only attempts to connect so it is not a problem.
Problem with bad post from QT so server does not understand request.

[Christian Ehrlicher]

@vicsoftware said in Prevent multiple TCP segments in post request:

QT trying send second post before closing connection. It's look like.

It does not send a http request a second time when you don't ask for it.

[vicsoftware]

@Christian-Ehrlicher Sorry but no. It is definitely QT behavior to resend request. I think. I am doing only one call:

const auto reply{ m_networkMan->post( request, data.toUtf8() ) };

It is pushButton handler and it has single debug std::cout message only before connection error emitted.

QT GET request works and sends only one time.

[Christian Ehrlicher]

Then please provide a minimal, reproducible example for us. I don't see why a http request should be sent twice when the other side does not request it.

[vicsoftware]

@Christian-Ehrlicher If server powered off then after longer time i got Request error: "Host unreachable" and there is no requests at all in wireshark.

[vicsoftware]

Why QT makes 2 tcp segments but other methods has 1 tcp segment. Someone can explain QT post behavior?

[Christian Ehrlicher]

@vicsoftware said in Prevent multiple TCP segments in post request:

Why QT makes 2 tcp segments but other methods has 1 tcp segment. Someone can explain QT post behavior?

Again: this has not much to do with Qt but with the underlying os. Maybe Qt writes the packet in two parts to the OS and curl only one. But this is completely irrelevant.

[vicsoftware]

@Christian-Ehrlicher And chrome extension only one. And python sends normal request. But QT does not. And sends it twice. And packet length wrong. Why?

[vicsoftware]

Have tried online post service, and same QT code sends only single post request with 2 tcp segments.

[vicsoftware]

This post is deleted!

[vicsoftware]

Simple curl code works:

const std::string url{ "http://" + m_addr + "/od/2400/01" };
        CURL *curl;
        CURLcode res;

        curl_global_init(CURL_GLOBAL_ALL);

        curl = curl_easy_init();
        if(curl) {
            curl_easy_setopt(curl, CURLOPT_URL, url.c_str() );
            curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "\"00000001\"");

            res = curl_easy_perform(curl);
            if(res != CURLE_OK)
                fprintf(stderr, "curl_easy_perform() failed: %s\n",
                        curl_easy_strerror(res));

            curl_easy_cleanup(curl);
        }
        curl_global_cleanup();

QNetworkManager post failed.

[J.Hilk]

@vicsoftware why do you post the code that "works" for you and not the one that doesn't!?!?!