Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. Does QAbstractSocket's internal write buffer abstract away writing data in chunks?

Does QAbstractSocket's internal write buffer abstract away writing data in chunks?

Scheduled Pinned Locked Moved Solved General and Desktop
4 Posts 2 Posters 279 Views
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • oblivioncthO Offline
    oblivioncthO Offline
    oblivioncth
    wrote on last edited by oblivioncth
    #1

    Pretty basic question, but I also have near zero experience when it comes to socket programming.

    I know that depending on multiple factors (like if the socket is blocking or not), data isn't always all sent at once when writing over a socket and one may need to do something like this to write the entirety of a message:

    int sock = 52; // Dummy socket
    const char* data = "Some ANSI data";
    size_t dataSize = strlen(data);
    size_t bytesWritten = 0;
    
    // Doesn't check for -1
    while(bytesWritten != dataSize)
        bytesWritten += send(sock, data + bytesWritten, dataSize - bytesWritten, 0) 
    

    My question is: Does the internal write buffer take care of this for me?

    I've seen a good amount of example code using QAbstactSocket derivatives that don't check the value returned by write, or only for the case of -1, and almost always when one sees this it's because its simple example code that doesn't have all the bells and whistles and doesn't handle all cases; however, from what I can gather it seems like that QAbstractSocket might indeed abstract this away so that I can simply call write(...) and know that the entire message will be queued up into the internal buffer as long as I don't overflow it (and I think somewhere I saw that it's 2GB in size) and it takes care of sending the message in the pieces that end up being required by the underlying system.

    It is my understanding that I can then monitor the "chunks" the messages are actually sent in via bytesWritten(qint64 bytes) and know that the entire message has been sent out (but not necessarily received) once bytesToWrite() returns 0 (assuming more hadn't been added to the buffer in the meanwhile).

    Is this true? If so, in what cases, if any, should I expect QAbstractSocket's (I'm actually using QTcpSocket if that matters) implementation of write et. al. to return less than the message size but not -1 if it always just throws the message into the buffer? Or is that more-or-less a non-occurrence in this particular derivative of QIODevice?

    Or is this all wrong and even though there is a buffer, write() will return the number of bytes queued to be sent in the next "chunk" and I do need to queue multiple writes until the message is fully in the buffer?

    P.S. I'm going to make my implementation utilize the event loop, but it's overkill since my use case is briefly acting as a client by always sending 2 tiny messages, checking that the responses to them are not errors, and then immediately disconnecting. I had originally intended to do this synchronously with very small timeouts for this reason, but this is for Windows and I think I'm not the only one that noticed all the warnings about the waitFor... functions being broken on it. Not that I envy the people that will hopefully stepup and fix the issue, but holy cow it's been like this for a decade at this point QTBUG-24451. Is the underlying asynchronous socket API for windows just that unreliable? Or is the implementation of QAbstractSocket just really mature and hard to rework in order to deal with issues specific to this platform that were realized later?

    Christian EhrlicherC 1 Reply Last reply
    0
    • oblivioncthO oblivioncth

      @Christian-Ehrlicher

      Sorry, when I said "utilize the event loop", I meant the existing one running in the thread, not nested ones I'm creating to artificially block. I meant it to specify that I am not doing the latter and am keeping things asynchronous, though I see now how that could be confusing.

      Ok then it sounds like a huge majority of the time, at least with reasonably sized messages, that I can safely write assuming all data will at least make it into the Qt buffer, but that nonetheless it is still theoretically possible in some circumstances this won't happen and therefore ideally should be accounted for.

      Thanks.

      Christian EhrlicherC Offline
      Christian EhrlicherC Offline
      Christian Ehrlicher
      Lifetime Qt Champion
      wrote on last edited by
      #4

      @oblivioncth said in Does QAbstractSocket's internal write buffer abstract away writing data in chunks?:

      Ok then it sounds like a huge majority of the time, at least with reasonably sized messages, that I can safely write assuming all data will at least make it into the Qt buffer, but that nonetheless it is still theoretically possible in some circumstances this won't happen and therefore ideally should be accounted for.

      Yes, simply check for the return value and compare the size. Write an error log if something went wrong so in the rare case where you get an error you at least know what went wrong.

      Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
      Visit the Qt Academy at https://academy.qt.io/catalog

      1 Reply Last reply
      1
      • oblivioncthO oblivioncth

        Pretty basic question, but I also have near zero experience when it comes to socket programming.

        I know that depending on multiple factors (like if the socket is blocking or not), data isn't always all sent at once when writing over a socket and one may need to do something like this to write the entirety of a message:

        int sock = 52; // Dummy socket
        const char* data = "Some ANSI data";
        size_t dataSize = strlen(data);
        size_t bytesWritten = 0;
        
        // Doesn't check for -1
        while(bytesWritten != dataSize)
            bytesWritten += send(sock, data + bytesWritten, dataSize - bytesWritten, 0) 
        

        My question is: Does the internal write buffer take care of this for me?

        I've seen a good amount of example code using QAbstactSocket derivatives that don't check the value returned by write, or only for the case of -1, and almost always when one sees this it's because its simple example code that doesn't have all the bells and whistles and doesn't handle all cases; however, from what I can gather it seems like that QAbstractSocket might indeed abstract this away so that I can simply call write(...) and know that the entire message will be queued up into the internal buffer as long as I don't overflow it (and I think somewhere I saw that it's 2GB in size) and it takes care of sending the message in the pieces that end up being required by the underlying system.

        It is my understanding that I can then monitor the "chunks" the messages are actually sent in via bytesWritten(qint64 bytes) and know that the entire message has been sent out (but not necessarily received) once bytesToWrite() returns 0 (assuming more hadn't been added to the buffer in the meanwhile).

        Is this true? If so, in what cases, if any, should I expect QAbstractSocket's (I'm actually using QTcpSocket if that matters) implementation of write et. al. to return less than the message size but not -1 if it always just throws the message into the buffer? Or is that more-or-less a non-occurrence in this particular derivative of QIODevice?

        Or is this all wrong and even though there is a buffer, write() will return the number of bytes queued to be sent in the next "chunk" and I do need to queue multiple writes until the message is fully in the buffer?

        P.S. I'm going to make my implementation utilize the event loop, but it's overkill since my use case is briefly acting as a client by always sending 2 tiny messages, checking that the responses to them are not errors, and then immediately disconnecting. I had originally intended to do this synchronously with very small timeouts for this reason, but this is for Windows and I think I'm not the only one that noticed all the warnings about the waitFor... functions being broken on it. Not that I envy the people that will hopefully stepup and fix the issue, but holy cow it's been like this for a decade at this point QTBUG-24451. Is the underlying asynchronous socket API for windows just that unreliable? Or is the implementation of QAbstractSocket just really mature and hard to rework in order to deal with issues specific to this platform that were realized later?

        Christian EhrlicherC Offline
        Christian EhrlicherC Offline
        Christian Ehrlicher
        Lifetime Qt Champion
        wrote on last edited by
        #2

        @oblivioncth said in Does QAbstractSocket's internal write buffer abstract away writing data in chunks?:

        I'm going to make my implementation utilize the event loop,

        Don't do this.

        check for the return value of QIODevice::write() will most likely return something != -1 only when it's internal and OS buffers are full (or when it's an unbuffered iodevice and the os can#t write all at once).
        But I would check the return value and try to send the missing stuff directly afterwards. Esp. without an eventloop - it will most likely kill your event handling,

        Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
        Visit the Qt Academy at https://academy.qt.io/catalog

        oblivioncthO 1 Reply Last reply
        3
        • Christian EhrlicherC Christian Ehrlicher

          @oblivioncth said in Does QAbstractSocket's internal write buffer abstract away writing data in chunks?:

          I'm going to make my implementation utilize the event loop,

          Don't do this.

          check for the return value of QIODevice::write() will most likely return something != -1 only when it's internal and OS buffers are full (or when it's an unbuffered iodevice and the os can#t write all at once).
          But I would check the return value and try to send the missing stuff directly afterwards. Esp. without an eventloop - it will most likely kill your event handling,

          oblivioncthO Offline
          oblivioncthO Offline
          oblivioncth
          wrote on last edited by oblivioncth
          #3

          @Christian-Ehrlicher

          Sorry, when I said "utilize the event loop", I meant the existing one running in the thread, not nested ones I'm creating to artificially block. I meant it to specify that I am not doing the latter and am keeping things asynchronous, though I see now how that could be confusing.

          Ok then it sounds like a huge majority of the time, at least with reasonably sized messages, that I can safely write assuming all data will at least make it into the Qt buffer, but that nonetheless it is still theoretically possible in some circumstances this won't happen and therefore ideally should be accounted for.

          Thanks.

          Christian EhrlicherC 1 Reply Last reply
          1
          • oblivioncthO oblivioncth

            @Christian-Ehrlicher

            Sorry, when I said "utilize the event loop", I meant the existing one running in the thread, not nested ones I'm creating to artificially block. I meant it to specify that I am not doing the latter and am keeping things asynchronous, though I see now how that could be confusing.

            Ok then it sounds like a huge majority of the time, at least with reasonably sized messages, that I can safely write assuming all data will at least make it into the Qt buffer, but that nonetheless it is still theoretically possible in some circumstances this won't happen and therefore ideally should be accounted for.

            Thanks.

            Christian EhrlicherC Offline
            Christian EhrlicherC Offline
            Christian Ehrlicher
            Lifetime Qt Champion
            wrote on last edited by
            #4

            @oblivioncth said in Does QAbstractSocket's internal write buffer abstract away writing data in chunks?:

            Ok then it sounds like a huge majority of the time, at least with reasonably sized messages, that I can safely write assuming all data will at least make it into the Qt buffer, but that nonetheless it is still theoretically possible in some circumstances this won't happen and therefore ideally should be accounted for.

            Yes, simply check for the return value and compare the size. Write an error log if something went wrong so in the rare case where you get an error you at least know what went wrong.

            Qt Online Installer direct download: https://download.qt.io/official_releases/online_installers/
            Visit the Qt Academy at https://academy.qt.io/catalog

            1 Reply Last reply
            1

            • Login

            • Login or register to search.
            • First post
              Last post
            0
            • Categories
            • Recent
            • Tags
            • Popular
            • Users
            • Groups
            • Search
            • Get Qt Extensions
            • Unsolved