Problems doing HTTP GET with QNetworkRequest

[dcoup]

I'm trying to use the get() method of the QNetworkRequest class to retrieve a small XML file from my company's web site. The file can be viewed with a browser with no problems. The Qt application gets a Qt UnknownContentError returned.

The browser's HTTP GET request looks like this:

GET /asn1ve/version.xml HTTP/1.1
Host: www.obj-sys.com
User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: __utma=128261458.1963682936.1358861953.1376483518.1376508822.171; __utmz=128261458.1358861953.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); PHPSESSID=9e5c245a6b212c1831d5b830e340e242; __utmb=128261458.2.10.1376508822; __utmc=128261458
Connection: keep-alive

The HTTP GET request that's sent by my Qt application looks like this:

GET /asn1ve/version.xml HTTP/1.1
Host: www.obj-sys.com
User-Agent: Mozilla/5.0
Accept: application/xml
Accept-Language: en-US,*
Accept-Encoding: gzip, deflate
Connection: keep-alive

The server's response to the Qt request is an HTTP 406 "Not Acceptable" error. The long text is "An appropriate representation of the requested resource could not be found on this server. This error was generated by Mod_Security."

I realize that the Qt HTTP GET isn't exactly the same as the browser HTTP GET, but I don't see any differences that would cause this error. The response to the browser request brings the file in with application/xml as the content type.

Any ideas?

[mtobin]

I've just spent most of the afternoon trying to work out a solution to this problem, and since I came across this thread describing the exact same problem, I thought I would post what I have found here, even though I hope you stumbled across the solution yourself by now.

The issue is entirely with the User-Agent. If you set the User Agent to anything different, the download will work. Upon experimenting, I cannot find any value that doesn't work other than the default "Mozilla/5.0" user agent.

request.setRawHeader("User-Agent","Fake User"); // Succeeds
request.setRawHeader("User-Agent","Mozillax/5.0"); // Succeeds
request.setRawHeader("User-Agent","Mozilla/5.0"); // Fails

I have no viable explanation for why this is happening.

Strangely, I didn't have this problem for several months of my application working correctly before it suddenly started to be an issue, which is a huge pain, because it means I have released software that can no longer access its auto-updater.

Matt

[mtobin]

As an update to this:

I got in touch with my web host, and in that case it turned out that the User-Agent Mozilla/5.0 was blocked by mod_security. Once this user agent was added to the white list, the default User Agent worked correctly.

It could be the server being accessed in the OP has a similar issue.

Matt

[dcoup]

In my case it seems that the software running on our web server didn't like a user agent specification that didn't have anything in parentheses. I was able to get it to work by specifying "Mozilla /5.0 (linux-gnu)" as the value for the user agent.

Assuming that the function I was trying to debug worked when it was originally written, I can only assume that our web hosting company changed something that broke it.

[rtavakko]

This is going to feel like a youtube comment...but thumbs up if you're still having this issue in 2019...
I didn't have exactly an issue with what the User-Agent was set to but the same thing happens if the raw header is not set at all.

Thanks for this thread guys!

[JonB]

@rtavakko
In my experience a lot of old stuff out there going back to 2009 can still be useful in Qt5 2019 :)