How to download file from a site without requiring any authentication, using QNetworkAccessManager

adnan

How to download file from a site without requiring any authentication, if the user is already signed in to the site? Are we supposed to pass cookies. How to do that? I am using firefox browser.

AcerExtensa

Use "QNetworkCookieJar":http://doc.qt.nokia.com/4.7-snapshot/qnetworkcookiejar.html for cookie handling on QNAM. You can set them with "setCookieJar":http://doc.qt.nokia.com/4.7-snapshot/qnetworkaccessmanager.html#setCookieJar function of your QNAM instance.

Or set them directly in header...

adnan

How can i load cookie from a file

AcerExtensa

Just read it from file with "QFile":http://doc.qt.nokia.com/4.7-snapshot/qfile.html and return them in "cookiesForUrl":http://doc.qt.nokia.com/4.7-snapshot/qnetworkcookiejar.html#cookiesForUrl function of your subclassed QNetworkCookieJar class.

adnan

This is the cookie which i have to pass to QNetworkAccessManager:
@S=gmail=gPwZoKGL8hgGA4ZHoej_Tg; S=gmail=xrWQCf29oq77myB3uSRKPg; @

    @request.setUrl(url);
      request.setAttribute(QNetworkRequest::HttpPipeliningAllowedAttribute, true);
      cookieJar->setCookiesFromUrl(QNetworkCookie::parseCookies(cookie),url);
      manager.setCookieJar(cookieJar);
      reply=manager.get(request);@

      The finished signal calls the following slot
     @int statusCode=0;
        statusCode=reply->attribute(QNetworkRequest::HttpStatusCodeAttribute).toInt();
       if(statusCode>300)
       {
           if(statusCode==307||reply->rawHeaderList().contains("Location"))
           {
            url=reply->header(QNetworkRequest::LocationHeader).toString();
            request.setUrl(url);
            request.setAttribute(QNetworkRequest::HttpPipeliningAllowedAttribute, true);
            cookieJar->setCookiesFromUrl(QNetworkCookie::parseCookies(cookie),url);
            manager.setCookieJar(cookieJar);
            reply=manager.get(request);
            return;
        }@

The download fails as it downloads the webpage instead of file. The url and cookies are provided by flashgot addon, firefox.

adnan

Am i committing any mistake in passing the cookies!

adnan

bq. Just read it from file with QFile [doc.qt.nokia.com] and return them in cookiesForUrl [doc.qt.nokia.com] function of your subclassed QNetworkCookieJar class.

Exact scenario is like this.
there is a site which require authentication. I want that after the user has logged into the site through browser, i want that my code should take that cookie from
hard disk and send a get request to portal with that cookie so that authentication is not required. I know the address of cookie file created on hard disk by browser. I wish to load that cookie in QNetworkCookieJar from the file whose location I know. Can you plz explain in detail.

AcerExtensa

Subclass QNetworkCookieJar:
@
class ExNetworkCookiesPrivate: public QNetworkCookieJar
{
public:
ExNetworkCookiesPrivate(QObject * parent = 0);

QList<QNetworkCookie> cookiesForUrl(const QUrl & url) const;
bool setCookiesFromUrl(const QList<QNetworkCookie> & cookieList, const QUrl & url);

private:
QMap<QString,QList<QNetworkCookie>> cookies;
};

ExNetworkCookiesPrivate::ExNetworkCookiesPrivate(QObject * parent):QNetworkCookieJar(parent)
{
//Here you can load your cookies from file
//insert it in this->cookie map with host(not the complete url) as an identificator
}

QList<QNetworkCookie> ExNetworkCookiesPrivate::cookiesForUrl(const QUrl & url) const
{
QList<QNetworkCookie> tmp = this->cookies.value(url.host());
return tmp;
}

bool ExNetworkCookiesPrivate::setCookiesFromUrl(const QList<QNetworkCookie> & cookieList, const QUrl & url)
{
//Check here if new cookie is already exist in the this->cookie variable
// this->cookie.contains(...) etc...
this->cookie.insert(url.host(), cookieList);
return true;
}
@

Set your cookies subclass right after you've created QNAM.
@
QNetworkAccessManager *manager = new QNetworkAccessManager();
manager->setCookieJar(new ExNetworkCookiesPrivate());
@

Some webpages use not only cookies for identification, they also can check headers for correct User-Agent, use session id as get parameter, use javascript to update cookies periodical(QNAM doesn't care about execution of javascript, use QScript for that)

You should check all this behaviors with wireshark for example. Sometimes it is easy to use QWebPage for such scenario...

adnan

THANKS A TON! It was really helpful of you giving such a deatiled answer.

adnan

In this code you haven't parsed the cookie. Isn't that required. The content of cookie file i use looks like this:
@.qt-project.org TRUE / FALSE 1349213044 exp_remember d25830fa076bd52ee591c35ae8a8234b
.qt-project.org TRUE / FALSE 1349213044 exp_expiration 1349213062
.bestvideodownloader.com TRUE / FALSE 1411009102 __utma 58991538.966392906.1347909513.1347909513.1347933303.2
.bestvideodownloader.com TRUE / FALSE 1347938902 __utmb 58991538.40.9.1347937102376
.bestvideodownloader.com TRUE / FALSE 1348319713472 __utmc 58991538@

"Cookie Parser":http://qt-project.org/forums/viewthread/3041

AcerExtensa

I didn't really understood you... It was just an example... Parsing was not required for me....

Just open your file with QFile, read it line by line and parse with "QNetworkCookie:parseCookies":http://qt-project.org/doc/qt-4.8/qnetworkcookie.html#parseCookies

adnan

This time i am passing cookies through a string, cookie
@ rPopHome=1; FF_JoinPromo=true; __utma=140252452.845088738.1348089486.1348089486.1348089486.1; __utmb=140252452.6.10.1348089486; __utmc=140252452; __utmz=140252452.1348089486.1.1.utmcsr=adf.ly|utmccn=(referral)|utmcmd=referral|utmcct=/7hYuB; ff_membership=SlyL0JODTZMIdfYHy/7imBJHqyHhO9ZHNE4bwnFPzoGYgPVEEneUMsfqaFDsqR+w9EotIx9i7PPL7ZIp1CiUSOtx0Y4mE1GmxayYmnJVD/z5aCTS2qT7fTItiqOprW8rlsEYw2Br7OoETaF3L1RGRQai1zlFQw6QO9qBEPKXC7rOx5LuEEKYlrZFKvuD/m7uDc6SWDJyaW67sRfo/bpb/Q==; ff_referrer_hash=o7e2qf0; @

@request.setUrl(url);
request.setAttribute(QNetworkRequest::HttpPipeliningAllowedAttribute, true);
cookieJar->setCookiesFromUrl(QNetworkCookie::parseCookies(cookie),url.host());
manager.setCookieJar(cookieJar);
reply=manager.get(request);@

But i get the following error, the downloaded file contains this message:
@<h1>HTTP/1.0 401 Unauthorized</h1>
<p>You have attempted to download via a Premium download link, but we could not verify your Premium account details.</p>
<p></h1>
@

I changed the user agent also, but it still failed:
@request.setRawHeader("User-Agent", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0.1");@

AcerExtensa

I've told you:

bq. Some webpages use not only cookies for identification, they also can check headers for correct User-Agent, use session id as get parameter, use javascript to update cookies periodical(QNAM doesn’t care about execution of javascript, use QScript for that)

bq. You should check all this behaviors with wireshark for example. Sometimes it is easy to use QWebPage for such scenario…

adnan

The download is successful if i pass username and password when @authenticationRequired(QNetworkReply*,QAuthenticator*)@ signal is emitted. But it fails with cookies. I used correct user-agent, most recent cookie generated at request time only. It didn't work.
bq. use session id as get parameter

Can u explain this a bit. Only this part is left to be included.

AcerExtensa

Some webpages use cookie + sessid methods together...
for example:

you opens http://webpage.web
inserts your username and password
new url will be http://webpage.web?id=md5summ

Just use wireshark to compare booth of requests, your and one from web-browser... Do it character-by-character! Don't tell they are similar... They are not!