Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. How to limit upload speed?
Forum Updated to NodeBB v4.3 + New Features

How to limit upload speed?

Scheduled Pinned Locked Moved Unsolved General and Desktop
8 Posts 2 Posters 3.3k Views 2 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • T Offline
    T Offline
    TheSHEEEP
    wrote on 29 Jun 2017, 07:59 last edited by TheSHEEEP
    #1

    Hello,

    I am using QNetworkAccessManager to upload a file (and some data) like this:

    QNetworkRequest request(<some URL>);
    QHttpMultiPart *multiPart = new QHttpMultiPart(QHttpMultiPart::FormDataType);
    <multipart is filled with single parts>
    
    networkAccessManager->post(request, multiPart);
    

    It works well.
    However, users have reported that the upload influences their upload bandwidth. Which... makes sense, of course ;)

    Now, I would like to implement something to limit the upload speed of this post request. However, after looking around for a while, I found absolutely nothing to do this in any simple way.

    There is setReadBufferSize for the QNetworkReply, and I read that this can be used to limit the download speed. But the description doesn't make it look like it would work for uploads as well. Also, there is a years old bug report for that function which doesn't exactly inspire confidence in it working at all (QTBUG-15065).

    Is there any way to achieve an upload throttle at all without having to rewrite Qt sources?
    A way to "globally" limit the upload speed of the application would be acceptable as well.

    I have found this ancient thing: https://doc.qt.io/archives/qq/qq17-ratecontrol.html
    It looks like that could be used, but since I do not only transfer binary data, but all the other HTML multi part stuff as well, it would mean a significant amount of work to pull it off correctly.

    1 Reply Last reply
    0
    • T Offline
      T Offline
      TheSHEEEP
      wrote on 4 Jul 2017, 08:35 last edited by
      #2

      After trying for many days, I have to conclude that limiting upload speed simply isn't possible.
      Even by heavily adjusting the torrent example with the rate control class, it cannot work.

      The main problem is that QIODevice's read & write functions are not virtual, you can overwrite QNetworkReply as much as you want, it won't help when it comes to upload limiting. Your own read/write functions won't be called and there are no signals that would allow you to interrupt the writing process.

      For download, there is at least the setReadBufferSize, and it works not perfectly (for some not at all, it seems), but it is there.
      Due to a lack of a setWriteBufferSize function, it is simply not possible.

      Other than by re-implementing the complete network side yourself, that is. Or changing Qt's sources, which is just impractical for many reasons.

      Should I make a report or feature request for this?

      K 1 Reply Last reply 4 Jul 2017, 09:03
      0
      • T TheSHEEEP
        4 Jul 2017, 08:35

        After trying for many days, I have to conclude that limiting upload speed simply isn't possible.
        Even by heavily adjusting the torrent example with the rate control class, it cannot work.

        The main problem is that QIODevice's read & write functions are not virtual, you can overwrite QNetworkReply as much as you want, it won't help when it comes to upload limiting. Your own read/write functions won't be called and there are no signals that would allow you to interrupt the writing process.

        For download, there is at least the setReadBufferSize, and it works not perfectly (for some not at all, it seems), but it is there.
        Due to a lack of a setWriteBufferSize function, it is simply not possible.

        Other than by re-implementing the complete network side yourself, that is. Or changing Qt's sources, which is just impractical for many reasons.

        Should I make a report or feature request for this?

        K Offline
        K Offline
        kshegunov
        Moderators
        wrote on 4 Jul 2017, 09:03 last edited by
        #3

        @TheSHEEEP said in How to limit upload speed?:

        The main problem is that QIODevice's read & write functions are not virtual

        They are, the relevant ones are protected: QIODevice::readData and QIODevice::writeData, however this might not be the thing to do in your case.

        Should I make a report or feature request for this?

        Yes, posting a feature request is what I'd do.

        Read and abide by the Qt Code of Conduct

        1 Reply Last reply
        1
        • T Offline
          T Offline
          TheSHEEEP
          wrote on 4 Jul 2017, 12:52 last edited by TheSHEEEP 7 Apr 2017, 12:56
          #4

          @kshegunov said in How to limit upload speed?:

          @TheSHEEEP said in How to limit upload speed?:

          The main problem is that QIODevice's read & write functions are not virtual

          They are, the relevant ones are protected: QIODevice::readData and QIODevice::writeData, however this might not be the thing to do in your case.

          Huh! Indeed they are. Must've looked int the wrong place...
          So maybe this is possible after all.

          What I am doing is using a class derived from QNetworkAccessManager, where I overwrote the createReply method like this:

          QNetworkReply*
          LimitingNetworkAccessManager::createRequest(QNetworkAccessManager::Operation op,
                                                      const QNetworkRequest& req,
                                                      QIODevice* outgoingData)
          {
              // Create our custom reply (by passing on the reply created by original Qt)
              QNetworkReply* qtReply = QNetworkAccessManager::createRequest(op, req, outgoingData);
              LimitingNetworkReply* reply = new LimitingNetworkReply(qtReply, this);
          
              // Add this reply to the limiter
              _limiter->addReplyToLimit(reply);
          
              // Return
              return reply;
          }
          

          However, my own LimitingNetworkReply class' read/writeData functions are never called. Despite this createRequest function returning my own reply. How is that possible?
          Is the original QNetworkReply created by QNetworkAccessManager::createRequest still being used for reading/writing?

          I knew using the original QNetworkAccessManager::createRequest function was risky here. But I just see no way of implementing my own QNetworkReply creation fully, since the original function is stock full with D-pointers and usage of private classes, so that's not something I can do when deriving.

          I feel like I'd have to completely write my own networking classes when there really should be a simple functionality to limit upload speed - which would be very easy to do by just using a timer when sending data over sockets...

          K 1 Reply Last reply 5 Jul 2017, 11:34
          0
          • T TheSHEEEP
            4 Jul 2017, 12:52

            @kshegunov said in How to limit upload speed?:

            @TheSHEEEP said in How to limit upload speed?:

            The main problem is that QIODevice's read & write functions are not virtual

            They are, the relevant ones are protected: QIODevice::readData and QIODevice::writeData, however this might not be the thing to do in your case.

            Huh! Indeed they are. Must've looked int the wrong place...
            So maybe this is possible after all.

            What I am doing is using a class derived from QNetworkAccessManager, where I overwrote the createReply method like this:

            QNetworkReply*
            LimitingNetworkAccessManager::createRequest(QNetworkAccessManager::Operation op,
                                                        const QNetworkRequest& req,
                                                        QIODevice* outgoingData)
            {
                // Create our custom reply (by passing on the reply created by original Qt)
                QNetworkReply* qtReply = QNetworkAccessManager::createRequest(op, req, outgoingData);
                LimitingNetworkReply* reply = new LimitingNetworkReply(qtReply, this);
            
                // Add this reply to the limiter
                _limiter->addReplyToLimit(reply);
            
                // Return
                return reply;
            }
            

            However, my own LimitingNetworkReply class' read/writeData functions are never called. Despite this createRequest function returning my own reply. How is that possible?
            Is the original QNetworkReply created by QNetworkAccessManager::createRequest still being used for reading/writing?

            I knew using the original QNetworkAccessManager::createRequest function was risky here. But I just see no way of implementing my own QNetworkReply creation fully, since the original function is stock full with D-pointers and usage of private classes, so that's not something I can do when deriving.

            I feel like I'd have to completely write my own networking classes when there really should be a simple functionality to limit upload speed - which would be very easy to do by just using a timer when sending data over sockets...

            K Offline
            K Offline
            kshegunov
            Moderators
            wrote on 5 Jul 2017, 11:34 last edited by
            #5

            @TheSHEEEP said in How to limit upload speed?:

            Is the original QNetworkReply created by QNetworkAccessManager::createRequest still being used for reading/writing?

            That'd be my guess, yes. The QNetworkAccessManager::createRequest method is rather complex and does all kinds of stuff, so calling it like this probably connects the created (Qt) reply object and thus your own object's methods aren't invoked.

            I feel like I'd have to completely write my own networking classes when there really should be a simple functionality to limit upload speed - which would be very easy to do by just using a timer when sending data over sockets...

            Well, the thing is these classes are "high-level", meaning they are there to simplify your life most of the time. Incidentally this also means that they're designed with a specific use case, and modifying that isn't at all trivial. For http NAM and friends thread the operations internally (and there are a lot of support classes that are used privately) so I'm not convinced you'd be able to do what you want in a simple fashion.

            Off the top of my head, a simple, albeit not exactly nice, workaround would be to connect the downloadProgress signal from the reply to a dedicated thread that manages the reception speed through Qt::BlockingQueuedConnection and intentionally blocking the event loop for some specific time. Depending on the internal implementation of QNAM it may provide ability to control the rate at which the download proceeds, however the best solution is still going to be to post a feature request.

            Read and abide by the Qt Code of Conduct

            T 1 Reply Last reply 7 Jul 2017, 10:32
            1
            • K kshegunov
              5 Jul 2017, 11:34

              @TheSHEEEP said in How to limit upload speed?:

              Is the original QNetworkReply created by QNetworkAccessManager::createRequest still being used for reading/writing?

              That'd be my guess, yes. The QNetworkAccessManager::createRequest method is rather complex and does all kinds of stuff, so calling it like this probably connects the created (Qt) reply object and thus your own object's methods aren't invoked.

              I feel like I'd have to completely write my own networking classes when there really should be a simple functionality to limit upload speed - which would be very easy to do by just using a timer when sending data over sockets...

              Well, the thing is these classes are "high-level", meaning they are there to simplify your life most of the time. Incidentally this also means that they're designed with a specific use case, and modifying that isn't at all trivial. For http NAM and friends thread the operations internally (and there are a lot of support classes that are used privately) so I'm not convinced you'd be able to do what you want in a simple fashion.

              Off the top of my head, a simple, albeit not exactly nice, workaround would be to connect the downloadProgress signal from the reply to a dedicated thread that manages the reception speed through Qt::BlockingQueuedConnection and intentionally blocking the event loop for some specific time. Depending on the internal implementation of QNAM it may provide ability to control the rate at which the download proceeds, however the best solution is still going to be to post a feature request.

              T Offline
              T Offline
              TheSHEEEP
              wrote on 7 Jul 2017, 10:32 last edited by
              #6

              @kshegunov That sounds a bit hackish, indeed, and I don't think it would work. Wouldn't that lead to rather unpredictable phases of high upload speed vs none instead of having a nice average upload speed?

              Either way, I did create a feature request and "solved" the problem in the meantime by using libcurl instead of QNetworkAccessManager.

              K 1 Reply Last reply 7 Jul 2017, 10:59
              3
              • T TheSHEEEP
                7 Jul 2017, 10:32

                @kshegunov That sounds a bit hackish, indeed, and I don't think it would work. Wouldn't that lead to rather unpredictable phases of high upload speed vs none instead of having a nice average upload speed?

                Either way, I did create a feature request and "solved" the problem in the meantime by using libcurl instead of QNetworkAccessManager.

                K Offline
                K Offline
                kshegunov
                Moderators
                wrote on 7 Jul 2017, 10:59 last edited by kshegunov 7 Jul 2017, 11:00
                #7

                @TheSHEEEP said in How to limit upload speed?:

                Wouldn't that lead to rather unpredictable phases of high upload speed vs none instead of having a nice average upload speed?

                Possibly, it was the first thing I could think of.

                PS.
                Please post the ticket here, so others searching for such a solution could find it through this thread.

                Read and abide by the Qt Code of Conduct

                1 Reply Last reply
                0
                • T Offline
                  T Offline
                  TheSHEEEP
                  wrote on 10 Jul 2017, 12:17 last edited by
                  #8

                  Sorry, I forgot to post the link:
                  https://bugreports.qt.io/browse/QTBUG-61787

                  1 Reply Last reply
                  1

                  1/8

                  29 Jun 2017, 07:59

                  • Login

                  • Login or register to search.
                  1 out of 8
                  • First post
                    1/8
                    Last post
                  0
                  • Categories
                  • Recent
                  • Tags
                  • Popular
                  • Users
                  • Groups
                  • Search
                  • Get Qt Extensions
                  • Unsolved