How to ensure fully asynchronous web requests?



  • I'm noticing some blocking or waiting when I issue simultaneous web requests using QWebFrame.load for urls which contain external resources (e.g. stylesheets). These requests are issued via an HTTP Web Server which accepts a url and returns a thumbnail, the source of which can be found "here.":https://github.com/agschwender/python-web2image-server/blob/master/web2imageserver/httpdaemon.py (It's written using PyQt, but I suspect, based on nothing, that this issue originates in Qt, but if I'm wrong, please correct me)

    The initial request to the HTML document happens fully asynchronously, but there appears to be waiting when it comes to requesting the corresponding CSS documents. Here's the response from a sleeping server which sleeps 3 seconds before returning an HTML document which includes 2 CSS links. The CSS documents sleep 1 second before returning their contents.

    @
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3004.36ms
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3001.36ms
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3001.17ms
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3001.41ms
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3002.15ms
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3002.40ms
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3001.79ms
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3001.30ms
    [I 110427 00:54:07 web:1235] 200 GET / (127.0.0.1) 3001.10ms
    [I 110427 00:54:08 web:1235] 200 GET / (127.0.0.1) 3001.61ms
    [I 110427 00:54:08 web:1235] 200 GET /main.css?v=KAoMBm (127.0.0.1) 1004.54ms
    [I 110427 00:54:08 web:1235] 200 GET /alt.css?v=KAoMBm (127.0.0.1) 1003.76ms
    [I 110427 00:54:08 web:1235] 200 GET /main.css?v=ZBVpqz (127.0.0.1) 1002.26ms
    [I 110427 00:54:08 web:1235] 200 GET /alt.css?v=ZBVpqz (127.0.0.1) 1001.73ms
    [I 110427 00:54:09 web:1235] 200 GET /main.css?v=xyebDC (127.0.0.1) 1003.25ms
    [I 110427 00:54:09 web:1235] 200 GET /alt.css?v=xyebDC (127.0.0.1) 1000.32ms
    [I 110427 00:54:09 web:1235] 200 GET /main.css?v=sOjwMX (127.0.0.1) 1000.47ms
    [I 110427 00:54:09 web:1235] 200 GET /alt.css?v=sOjwMX (127.0.0.1) 1002.08ms
    [I 110427 00:54:10 web:1235] 200 GET /main.css?v=sbjacs (127.0.0.1) 1001.72ms
    [I 110427 00:54:10 web:1235] 200 GET /alt.css?v=sbjacs (127.0.0.1) 1000.32ms
    [I 110427 00:54:10 web:1235] 200 GET /main.css?v=WgidrV (127.0.0.1) 1000.58ms
    [I 110427 00:54:10 web:1235] 200 GET /alt.css?v=WgidrV (127.0.0.1) 1002.89ms
    [I 110427 00:54:11 web:1235] 200 GET /main.css?v=dTGTQS (127.0.0.1) 1003.70ms
    [I 110427 00:54:11 web:1235] 200 GET /alt.css?v=dTGTQS (127.0.0.1) 1003.28ms
    [I 110427 00:54:11 web:1235] 200 GET /main.css?v=DCDzgA (127.0.0.1) 1000.32ms
    [I 110427 00:54:11 web:1235] 200 GET /alt.css?v=DCDzgA (127.0.0.1) 1001.39ms
    [I 110427 00:54:12 web:1235] 200 GET /main.css?v=nDGaXE (127.0.0.1) 1001.68ms
    [I 110427 00:54:12 web:1235] 200 GET /alt.css?v=nDGaXE (127.0.0.1) 1000.30ms
    [I 110427 00:54:12 web:1235] 200 GET /main.css?v=EacZOY (127.0.0.1) 1000.21ms
    [I 110427 00:54:12 web:1235] 200 GET /alt.css?v=EacZOY (127.0.0.1) 1000.94ms
    @

    The server only seems to handle two webframes concurrently and waits until 1 is complete before moving to the next waiting webframe. Is there any way to stop the webframes for waiting while they load the external resources, or, if not, increase the number of webframes that are handled concurrently? Thanks in advance.



  • I think that QnetworkAccessManager is limited to downloading 6 requests at once (like most web browsers). I believe that this is hard-coded in Qt so you would have to hack that and rebuild Qt using your new value. That could be what is limiting you.



  • The only reason I'm hesitant to agree with that, is that the initial request doesn't seem to be affected by that limitation. Also, each initial request is made with its own instance of the QNetworkAccessManager class.

    I also followed the quick and dirty instructions described in "Increasing the maximum number of connections per host of QtWebKit":http://www.techques.com/question/1-4879919/Increasing-the-maximum-number-of-connections-per-host-of-QtWebKit which didn't result in any noticeable improvement.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.