Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Prevent multiple TCP segments in post request



  • Hi,

    I tried post request to server.

    curl and chrome post extension works fine:

    curl -X POST -d "\"00000001\"=" -H "Content-Length: 10" http://192.168.0.200/od/2400/01 -v
    
    Frame 524: 222 bytes on wire (1776 bits), 222 bytes captured (1776 bits) on interface 0
    Ethernet II, Src: 48:b0:2d:05:a8:be (48:b0:2d:05:a8:be), Dst: NanotecE_00:43:16 (44:aa:e8:00:43:16)
    Internet Protocol Version 4, Src: 192.168.0.110, Dst: 192.168.0.200
    Transmission Control Protocol, Src Port: 37720, Dst Port: 80, Seq: 1, Ack: 1, Len: 168
        Source Port: 37720
        Destination Port: 80
        [Stream index: 16]
        [TCP Segment Len: 168]
        Sequence number: 1    (relative sequence number)
        [Next sequence number: 169    (relative sequence number)]
        Acknowledgment number: 1    (relative ack number)
        0101 .... = Header Length: 20 bytes (5)
        Flags: 0x018 (PSH, ACK)
        Window size value: 29200
        [Calculated window size: 29200]
        [Window size scaling factor: -2 (no window scaling used)]
        Checksum: 0x8349 [unverified]
        [Checksum Status: Unverified]
        Urgent pointer: 0
        [SEQ/ACK analysis]
        [Timestamps]
        TCP payload (168 bytes)
        TCP segment data (1 byte)
    Hypertext Transfer Protocol
        POST /od/2400/01 HTTP/1.1\r\n
        Host: 192.168.0.200\r\n
        User-Agent: curl/7.58.0\r\n
        Accept: */*\r\n
        Content-Length: 10\r\n
        Content-Type: application/x-www-form-urlencoded\r\n
        \r\n
        [Full request URI: http://192.168.0.200/od/2400/01]
        [HTTP request 1/1]
        [Response in frame: 525]
        File Data: 10 bytes
    HTML Form URL Encoded: application/x-www-form-urlencoded
        Form item: ""00000001"" = ""
            Key: "00000001"
            Value:
    

    But QT code does not work (Request error: "Connection closed"):

    QString url{ "http://" + QString::fromStdString( m_addr ) + "/od/2400/01" };
    QNetworkRequest request{ QUrl( url ) };
    request.setHeader( QNetworkRequest::ContentTypeHeader, "application/x-www-form-urlencoded" );
    request.setHeader( QNetworkRequest::ContentLengthHeader, 10 );
    QString data{ "\"00000002\"" };
    const auto reply{ m_networkMan->post( request, data.toUtf8() ) };
    
    Frame 422: 64 bytes on wire (512 bits), 64 bytes captured (512 bits) on interface 0
    Ethernet II, Src: 48:b0:2d:05:a8:be (48:b0:2d:05:a8:be), Dst: NanotecE_00:43:16 (44:aa:e8:00:43:16)
    Internet Protocol Version 4, Src: 192.168.0.110, Dst: 192.168.0.200
    Transmission Control Protocol, Src Port: 37718, Dst Port: 80, Seq: 227, Ack: 1, Len: 10
        Source Port: 37718
        Destination Port: 80
        [Stream index: 15]
        [TCP Segment Len: 10]
        Sequence number: 227    (relative sequence number)
        [Next sequence number: 237    (relative sequence number)]
        Acknowledgment number: 1    (relative ack number)
        0101 .... = Header Length: 20 bytes (5)
        Flags: 0x018 (PSH, ACK)
        Window size value: 29200
        [Calculated window size: 29200]
        [Window size scaling factor: -2 (no window scaling used)]
        Checksum: 0x82ab [unverified]
        [Checksum Status: Unverified]
        Urgent pointer: 0
        [SEQ/ACK analysis]
        [Timestamps]
        TCP payload (10 bytes)
        TCP segment data (10 bytes)
    [2 Reassembled TCP Segments (236 bytes): #420(226), #422(10)]
        [Frame: 420, payload: 0-225 (226 bytes)]
        [Frame: 422, payload: 226-235 (10 bytes)]
        [Segment count: 2]
        [Reassembled TCP length: 236]
        [Reassembled TCP Data: 504f5354202f6f642f323430302f303120485454502f312e...]
    Hypertext Transfer Protocol
        POST /od/2400/01 HTTP/1.1\r\n
        Content-Type: application/x-www-form-urlencoded\r\n
        Content-Length: 10\r\n
        Connection: Keep-Alive\r\n
        Accept-Encoding: gzip, deflate\r\n
        Accept-Language: en-US,*\r\n
        User-Agent: Mozilla/5.0\r\n
        Host: 192.168.0.200\r\n
        \r\n
        [Full request URI: http://192.168.0.200/od/2400/01]
        [HTTP request 1/1]
        File Data: 10 bytes
    HTML Form URL Encoded: application/x-www-form-urlencoded
        Form item: ""00000001"" = ""
            Key: "00000001"
            Value:
    

    What i see is 2 tcp segments: headers and data.
    How to prevent tcp segments splitting?

    Best regards, Viktor.


  • Lifetime Qt Champion

    @vicsoftware said in Prevent multiple TCP segments in post request:

    How to prevent tcp segments splitting?

    You can't (unless you change the official protocol - maybe ask at ietf.org if they change it for you :) )

    I don't see why it should not work, you code does not match the wireshark dump (0000001 <-> 0000002)

    Maybe it has something against the user agent or whatever but the http packet has at least the same data so it can't be the reason.
    How do you receive the error?



  • @Christian-Ehrlicher TCP segments it is my assumption. It is differents what i see. Code value does not matter. Can be "0000000X".
    Error received after couple seconds, and wireshark shows that QT sent 2 similar post requests. May be it trying to reconnect.

    QT GET request works fine.

    Here is how i catch errors:

    m_networkMan = new QNetworkAccessManager();
    QObject::connect( m_networkMan, & QNetworkAccessManager::finished, this, [ = ]( QNetworkReply * reply )
       {
           if( reply->error() )
           {
               const auto error{ reply->errorString() };
               qDebug() << "Request error: " << error;            
               m_isConnected = false;
               return;
           }
           m_isConnected = true;
           const auto answer{ reply->readAll() };
           qDebug() << "Request answer: " << answer;       
       });
    

    Here is chrome extension output, it works like curl too:

    Frame 53: 185 bytes on wire (1480 bits), 185 bytes captured (1480 bits) on interface 0
    Ethernet II, Src: 48:b0:2d:05:a8:be (48:b0:2d:05:a8:be), Dst: NanotecE_00:43:16 (44:aa:e8:00:43:16)
    Internet Protocol Version 4, Src: 192.168.0.110, Dst: 192.168.0.200
    Transmission Control Protocol, Src Port: 39152, Dst Port: 80, Seq: 1, Ack: 1, Len: 131
        Source Port: 39152
        Destination Port: 80
        [Stream index: 7]
        [TCP Segment Len: 131]
        Sequence number: 1    (relative sequence number)
        [Next sequence number: 132    (relative sequence number)]
        Acknowledgment number: 1    (relative ack number)
        0101 .... = Header Length: 20 bytes (5)
        Flags: 0x018 (PSH, ACK)
        Window size value: 29200
        [Calculated window size: 29200]
        [Window size scaling factor: -2 (no window scaling used)]
        Checksum: 0x8324 [unverified]
        [Checksum Status: Unverified]
        Urgent pointer: 0
        [SEQ/ACK analysis]
        [Timestamps]
        TCP payload (131 bytes)
        TCP segment data (2 bytes)
    Hypertext Transfer Protocol
        POST /od/2400/01 HTTP/1.1\r\n
        HOST: 192.168.0.200\r\n
        content-type: application/x-www-form-urlencoded\r\n
        content-length: 10\r\n
        \r\n
        [Full request URI: http://192.168.0.200/od/2400/01]
        [HTTP request 1/1]
        [Response in frame: 54]
        File Data: 10 bytes
    HTML Form URL Encoded: application/x-www-form-urlencoded
        Form item: ""00000002"" = ""
            Key: "00000002"
            Value:
    

    Also if i using that:

    QUrlQuery postData;
    postData.addQueryItem( "\"00000002\"", "" );
    const auto reply{ m_networkMan->post( request, postData.query().toUtf8() ) };
    

    then QT sent only ' "0000000 ', so i am using inline QString data{ "\"00000002\"" };

    It is a bug?

    Server API waits Content-Length 10 bytes, Content-Type x-www-form-urlencoded and data "0000000X"=

    curl and chrome post works.


  • Lifetime Qt Champion

    As said - it depends on the OS or whatever how much tcp packets are sent for one request. It does not affect anything on the http protocol. The problem must be somewhere else.



  • It is Ubuntu 18, Nvidia Jetson AGX, Arm64

    Requests executes on same system, why curl and chrome sends single valid post request, but QT sends splitted segments and 2 requests?


  • Lifetime Qt Champion

    @vicsoftware said in Prevent multiple TCP segments in post request:

    but QT sends splitted segments and 2 requests?

    It does not send two requests. It sends one request with two tcp packets. Maybe it's how write() is called on the socket or the moon phase or whatever.



  • @Christian-Ehrlicher No. QT sends 2 request until "closed connection" error and every request has 2 TCP segments.
    Curl and chrome sends single request with 1 tcp segment.

    Here is 2 first QT request, third is chrome request forth is curl.
    Screenshot from 2021-10-12 14-05-50.png

    As we see length is 64 bytes for QT, and 185 for chrome and 222 for curl.

    But QT post has more headers and it shoud be longer bytes. It shows 64 bytes but:

    Screenshot from 2021-10-12 14-13-28.png

    Really tcp has 235 bytes.


  • Lifetime Qt Champion

    No. QT sends 2 request

    This is a new information.

    Qt only send two requests when you programmed it to do so.



  • @Christian-Ehrlicher No. I execute only single post from code. QT trying send second post before closing connection. It's look like.

    If it is only attempts to connect so it is not a problem.
    Problem with bad post from QT so server does not understand request.


  • Lifetime Qt Champion

    @vicsoftware said in Prevent multiple TCP segments in post request:

    QT trying send second post before closing connection. It's look like.

    It does not send a http request a second time when you don't ask for it.



  • @Christian-Ehrlicher Sorry but no. It is definitely QT behavior to resend request. I think. I am doing only one call:

    const auto reply{ m_networkMan->post( request, data.toUtf8() ) };
    

    It is pushButton handler and it has single debug std::cout message only before connection error emitted.

    QT GET request works and sends only one time.


  • Lifetime Qt Champion

    Then please provide a minimal, reproducible example for us. I don't see why a http request should be sent twice when the other side does not request it.



  • @Christian-Ehrlicher If server powered off then after longer time i got Request error: "Host unreachable" and there is no requests at all in wireshark.



  • Why QT makes 2 tcp segments but other methods has 1 tcp segment. Someone can explain QT post behavior?


  • Lifetime Qt Champion

    @vicsoftware said in Prevent multiple TCP segments in post request:

    Why QT makes 2 tcp segments but other methods has 1 tcp segment. Someone can explain QT post behavior?

    Again: this has not much to do with Qt but with the underlying os. Maybe Qt writes the packet in two parts to the OS and curl only one. But this is completely irrelevant.



  • @Christian-Ehrlicher And chrome extension only one. And python sends normal request. But QT does not. And sends it twice. And packet length wrong. Why?



  • Have tried online post service, and same QT code sends only single post request with 2 tcp segments.

    Screenshot from 2021-10-12 15-28-31.png



  • This post is deleted!


  • Simple curl code works:

    const std::string url{ "http://" + m_addr + "/od/2400/01" };
            CURL *curl;
            CURLcode res;
    
            curl_global_init(CURL_GLOBAL_ALL);
    
            curl = curl_easy_init();
            if(curl) {
                curl_easy_setopt(curl, CURLOPT_URL, url.c_str() );
                curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "\"00000001\"");
    
                res = curl_easy_perform(curl);
                if(res != CURLE_OK)
                    fprintf(stderr, "curl_easy_perform() failed: %s\n",
                            curl_easy_strerror(res));
    
                curl_easy_cleanup(curl);
            }
            curl_global_cleanup();
    

    QNetworkManager post failed.


  • Moderators

    @vicsoftware why do you post the code that "works" for you and not the one that doesn't!?!?!



  • @vicsoftware
    Two comments:

    • As @Christian-Ehrlicher has said, you need to produce (minimal, test) Qt code, not curl code, if you want someone to look at it as a Qt issue.

    • You, and your program, are never supposed to care what is going on at the TCP packet level. Whether an application makes a request which gets passed as one TCP packet, or two, or other, should have no effect. We have all used TCP for years, with Qt or other toolkits, and never cared about this or noticed any behaviour such as you report with "errors" occurring. Again, for the record, it is basically down to the OS/TCP level how it chooses to packet-ise messages. Artefacts such as Nagle's algorithm will commonly affect the runtime packet behaviour.



  • @J-Hilk QT code posted above.



  • @JonB I have nanotec.com tcp controller C5-E-2-81. It has REST API to control driver. So i can only get/post. And i can't reproduce for you it. You do not have that controller.
    I tried online service and it works. But that service do not dispatch input data, it only receives post so it is not the same as controller.

    In this case, the question is why all other methods do but QT does not? It simple post but it does not work.



  • Why does your curl data, "\"00000001\"=" and Qt data, "\"00000002\"" both have a content length of '10' (at least that's what you put in the request header for both)? Will you ever include a 'value' with your form request (i.e., don't form requests usually consist of key-value pairs)?



  • @mchinand Yes. It is only key value of 10bytes without value. Such controller REST API and recommendations from developers. Digit does not matter.



  • @vicsoftware said in Prevent multiple TCP segments in post request:

    @mchinand Yes. It is only key value of 10bytes without value. Such controller REST API and recommendations from developers. Digit does not matter.

    Right, but the length of the data strings is not the same yet you set the same content-length in the header for both of them.



  • @mchinand All working cases have string with 10 bytes length. Can you provide example of your idea?



  • @vicsoftware said in Prevent multiple TCP segments in post request:

    @mchinand All working cases have string with 10 bytes length. Can you provide example of your idea?

    from your post above:
    curl -X POST -d "\"00000001\"="
    and
    QString data{ "\"00000002\"" };



  • @mchinand
    Content length used by REST API to extract string from data.

    So first 10 bytes of these lines will have same content?
    curl -X POST -d "\"00000001\""
    and
    QString data{ "\"00000001\"" };



  • Here is from controller manual:

    Screenshot from 2021-10-13 00-41-03.png


Log in to reply