Getting the QNetworkRequest right

mikeosoft

I am using Qt 5.10.1 within Qt Creator 4.5.2 on Kubuntu 17.10

I'm trying to access the Oxford English dictionary to find information relating to specific words from within the application.

I have created an account with them and I can log into the website, see the account is active and I have an app_id and app_key defined by them that I'm using, though not the ones used within my code examples.

The problem I'm having is that I'm getting the following error:

Error transferring https://od-api.oxforddictionaries.com:443/api/v1/entries/en/spectrum - server replied: Forbidden

The commands work if I use curl to execute them and I'm using the same url and header information within my program as is used by curl.

Including or not including the port number 443, doesn't seem to make any difference.

The id and key are altered for the sake of showing the code, but it works with my real ones.

curl -X GET --header "Accept: application/json" --header "app_id: 5f5ddfdd" --header "app_key: a4a5410cea52a716086abae4563cb4" "https://od-api.oxforddictionaries.com/api/v1/entries/en/spectrum"

In my class constructor I have the line below and the signal/slot connection, which works and is what lets me see the error message.

m_network_access = new QNetworkAccessManager(this);

I have the code below in the method that I call to get the word definition and as I get an error that appears to be from the website I'm assuming that the error is in my formatting of the request.

QUrl url = "https://od-api.oxforddictionaries.com:443/api/v1/entries/en/" + word;
    QNetworkRequest request(url);
    request.setSslConfiguration(QSslConfiguration::defaultConfiguration());
    request.setHeader(QNetworkRequest::ContentTypeHeader, "application/json");
    
    // the id and key are altered for the sake of showing the code
    request.setRawHeader("app_id", "5f5ddfdd");
    request.setRawHeader("app_key", "a4a5410cea52a716086abae4563cb4");

    // This code is for debugging purposes and iterates the header elements and it shows three elements with the right names and values.
    QList<QByteArray> look = request.rawHeaderList();
    QByteArray value = "test";
    foreach(QByteArray name, look) {
       value = request.rawHeader(name);
    }

    m_network_access->get(request);

I use signals and slots to know when the response has been received and at that point I check first for an error and if there isn't one will read and process the Json file. So far I'm only getting the errors.

Googling the error message shows that the API doesn't support CORS, (Cross-origin resource sharing), which is defined by W3C and is a way that scripting within a browser, can make a request and receive the information back even though the browser sits between it and the server.

As this is a stand alone application, I don't think the lack of support for CORS is what is causing the problem, can you confirm this and let me know if there is anything obvious I am doing wrong.

Thanks.

shaan7

Not sure if it will actually fix your problem, but pointing out what I see-
You will not get --header "Accept: application/json" from
request.setHeader(QNetworkRequest::ContentTypeHeader, "application/json");

because ContentTypeHeader sets "Content-Type: application/json"

You should try this instead-
request.setRawHeader("Accept", "application/json");

mikeosoft

The --header parameter is part of the curl syntax rather than something that, QNetworkRequest would need me to insert - I think?

My assumption is that if I add the header elements to the QNetworkrequest object, the object will take care of formating them/building the request correctly.

When you use the API website it gives you examples of the curl commands as examples along with Python, swift, Objective-C and Java. I just used them to prove that it would work as part of my investigation and to show an example that works.

Thanks.

mikeosoft

I'll try it again with the accept and see if that works.

shaan7

I wasn't talking about --header, what I meant was that using ContentTypeHeader sets the Content-Type header, not Accept (which is mentioned in the curl example that you've told it works).

@mikeosoft said in Getting the QNetworkRequest right:

My assumption is that if I add the header elements to the QNetworkrequest object, the object will take care of formating them/building the request correctly.

That is correct.

mikeosoft

Yes, that was a good observation, but I'm afraid it doesn't work. I started off using the Accept version and then tried converting from the raw type to the standard type to see if that work. I just tried it again and I still get exactly the same response.

request.setRawHeader("Accept", "application/json");

Thanks for that.

shaan7

Try this for debugging, make a GET call to https://httpbin.org/headers which will respond back with the headers that you sent it. See if they look right.

SGaist

Hi,

Which version of OpenSSL do you have installed on your system ?
Do you get the same problem if you use your distribution provided Qt ?

mikeosoft

I got prompted for an upgrade to Qt Creator 4.6.0 this morning, so I'm now using that.

Trying the link to get the headers, gets me this, which to me look right in that it is formatted correctly and includes the 3 elements that I was specifically adding to the header.

{
  "headers": {
    "Accept": "application/json", 
    "Accept-Encoding": "gzip, deflate", 
    "Accept-Language": "en-GB,*", 
    "App-Id": "5f5ddfdd", 
    "App-Key": "a4a5410cea52a716086abae4563cb4", 
    "Connection": "close", 
    "Host": "httpbin.org", 
    "User-Agent": "Mozilla/5.0"
  }
}

Thanks for suggesting that link it was useful.

The OpenSSL version is:
OpenSSL 1.0.2g 1 Mar 2016

I'll try it with distribution version of Qt shortly and let you know what happens.

A number of people who reported this error were using CORS, which I assume I'm not using can you confirm that I'm not, just to eliminate that?

Thank you both of you.

mikeosoft

If I install the default version, which is Qt 5.9.1 with QT Creator 4.3.1 In my working machine I get the error

/usr/lib/x86_64-linux-gnu/libQt5Core.so.5: version `Qt_5.10' not found

I have been trying with a Virtual Machine, but Kubuntu seems to be a bit unstable and is giving me problems.

I installed it a little while ago after moving from Mint because they are giving up on KDE. On my desktop machine it has been very unstable, locking up if leaving it for a short period. It appears to relate to the graphics and I have tried every graphic driver variation and am currently using the one that worked the best.

I didn't have these problems with my laptop, but both seem to have a problem when you build a virtual-box virtual machine, it switches into an error screen resulting from a know Ubuntu error, which has a fix and tells you how to get out of it, but that doesn't work on these Virtual Machines. Which means I can't progress with building virtual machines to test this on.

Mint as a Ubuntu derivative also had the known error with the screen locking, but the fix took you straight out of the situation.

As I'm a bit stuck with that approach, do you have any other suggestions I can try.

Thanks.

shaan7

You might want to try KDE Neon although I really doubt your distribution is the problem here. If you are completely sure that the curl command line you originally posted still works then you should also point curl to https://httpbin.org/headers and compare the differences.
I really doubt CORS is the issue because you're not adding any Origin: headers in your curl command line and you claim it still works.

mikeosoft

Other than the User-Agent, which would be different because they are different tools these two were present in the Qt generated version, but not in the curl version.

"Accept-Encoding": "gzip, deflate", 
"Accept-Language": "en-GB,*",

I tried adding both the additional headers to the curl command and got the warning messages below, followed by the correct Json response. I tried each header individually and both at the same time.

curl: (6) Could not resolve host: en-GB,*
curl: (6) Could not resolve host: gzip, deflate

Neither stopped it happening with curl, but they do seem to stop the Qt version.

I looked for ways to remove these header elements and it looks like QNetworkRequest won't let you do it. See bug QTBUG-9463.

mikeosoft

I tried live booting from a KDE Neon DVD and running my debug executable from my SSD and the error was the same as when booting from my installed Kubuntu.

I am keen to stay with a Ubuntu based distribution because of a number of other tools that I couldn't get to work in opensuse tumbleweed and KAOS when I tried them from those distros.

shaan7

@mikeosoft said in Getting the QNetworkRequest right:

I am keen to stay with a Ubuntu based distribution

KDE Neon is based on Ubuntu :P

But in any case, as I said earlier, I doubt your problem is the distro.