download url with redirection

skylendar

Hi there and thx for reading and answering this post if you can.

I wish to download a file from dropbox from a qt app.

An example has been provided below using curl

curl -O -J -L "https://www.dropbox.com/scl/fo/y3naz62gy6f6qfrhquu7u/h/all_ast/ast_100_199/ast100/s100001s.se1?rlkey=ejltdhb262zglm7eo6yfj2940&dl=0"

Notice that this url is redirected towards another path (hence -L)

Let's try to make it with this basic qt snippet:

QNetworkAccessManage Manager;
QNetworkRequest request;
request.setUrl("https://www.dropbox.com/scl/fo/y3naz62gy6f6qfrhquu7u/h/all_ast/ast_100_199/ast100/s100001s.se1?rlkey=ejltdhb262zglm7eo6yfj2940&dl=0");
QNetworkReply *reply = Manager.get(request);
while (!reply->isFinished()) qApp->processEvents(QEventLoop::WaitForMoreEvents, 500);
QByteArray data = reply->readAll();

However data doesn't contain the wanted file. The redirection seems to
be ignored.

Any idea ?

Thx again.

Pl45m4

@skylendar

What Qt version are you using?

https://doc.qt.io/qt-6/network-changes-qt6.html#changes-in-qnetworkaccessmanager-s-default-behavior

Check:

https://doc.qt.io/qt-6/qnetworkrequest.html#RedirectPolicy-enum

skylendar

I'm using qt5.15

Pl45m4

@skylendar

The default value in 5.15 is:

ManualRedirectPolicy: not following any redirects.

https://doc.qt.io/qt-5/qnetworkrequest.html#RedirectPolicy-enum

Try QNetworkRequest::SameOriginRedirectPolicy

skylendar

will try, thx for the answer

skylendar

@Pl45m4 said in download url with redirection:

QNetworkRequest::SameOriginRedirectPolicy

Ok, I saw this enum, but it is unclear where to use it, because the QNetworkRequest header doesn't seem to provide any method using it.

skylendar

Oops, it is used by the networkaccessmanager, but by individual requests

Sorry...

skylendar

Ok, it's late, will try tomorrow

Pl45m4

@skylendar

Not tested:

    QNetworkAccessManager nam;
    nam.setRedirectPolicy(QNetworkRequest::SameOriginRedirectPolicy); // set on NAM for all further requests
    // or:
    QNetworkRequest req; // this request only
    req.setAttribute(QNetworkRequest::Attribute::RedirectPolicyAttribute,
                     QNetworkRequest::SameOriginRedirectPolicy);

skylendar

I tested it, reply->error() returns NoError, but I only got an empty file...

jsulm

@skylendar Please show your current code.

skylendar

#include <QtCore/QtCore>
#include <QtNetwork/QtNetwork>

int main(int argc, char *argv[])
{
    QCoreApplication app(argc, argv);

    QNetworkAccessManager manager;
		manager.setRedirectPolicy(QNetworkRequest::SameOriginRedirectPolicy);
		
    QUrl initialUrl("https://www.dropbox.com/scl/fo/y3naz62gy6f6qfrhquu7u/h/all_ast/ast_100_199/ast100/s100001s.se1?rlkey=ejltdhb262zglm7eo6yfj2940&dl=0");


    QNetworkRequest request;
   request.setUrl(initialUrl);
		request.setAttribute(QNetworkRequest::Attribute::RedirectPolicyAttribute,
                     QNetworkRequest::SameOriginRedirectPolicy);

		QNetworkReply *reply = manager.get(request);
		while (!reply->isFinished())        qApp->processEvents(QEventLoop::WaitForMoreEvents, 500);

		qDebug() << "erroe:" << reply->error() << "reply size:" <<    reply->size();
		QFile f("se00020s.se1");
		f.open(QIODevice::WriteOnly);
		f.write(reply->readAll());
		f.close();
                reply->deleteLater();
		
}

This code is compiled with the usual:

g++ -I /usr/include/qt5 redir1.cpp -lQt5Core -lQt5Network

Before, the code returned an empty file. Now, it contains an html page asking for a dropbox identification, not the wanted file.

Pl45m4

@skylendar

If your file is not public, you need to add your credentials to authenticate

skylendar

but the file is public. Anyone can download it !

Christian Ehrlicher

@skylendar said in download url with redirection:

but the file is public. Anyone can download it !

I need to enter some kind of credentials...

JonB

@skylendar Did you at least try visiting the Url from an Incognito window?

Pl45m4

@skylendar said in download url with redirection:

but the file is public. Anyone can download it !

Maybe you are right, but it seems that DropBox changed its policies and it's required to authenticate somehow...

https://www.dropboxforum.com/t5/Create-upload-and-share/New-URLs-for-shared-links-affecting-3rd-party-integrations/td-p/695197

This thread on DropBox forum says that it's not that easy anymore to share dropbox folders or files with anyone. :(
The link structure changed (type of file often not recognized anymore) and this authentication page was added

My favourite comment there:

One day someone got up there and decided to change the entire link structure in Dropbox. He didn't think that it might affect a lot of third-party apps, and in fact he didn't think about anything!

Which is very surprising for a serious company like Dropbox.

We already have a group of developers who are considering moving to Amazon's service.
In addition, they do not respond to support emails or opened tickets. Really disappointing, my whole app is just disabled.

Just disappointing and infuriating!

(source)

That seems about it! I know, security issue... etc. but I also know how DropBox was used to "quickly share a file" with a defined group or just with everyone who got that link.

skylendar

Ok folks, I thought I made a bug somewhere but this is a dropbox issue. Thx for your help

C.

skylendar

I'm back again, because I was told that the provided url was wrong. with:

https://www.dropbox.com/scl/fo/y3naz62gy6f6qfrhquu7u/h/all_ast/ast100/s100001s.se1?rlkey=ejltdhb262zglm7eo6yfj2940&dl=0

I got the right file with curl but not with the qt test code, but this time, I got this:

.se1 files are supported but something went wrong.

and download is proposed via a sign up.

skylendar

I also added :

request.setRawHeader("User-Agent", "curl/7.54.1");

but this time. my test utility returned:

QNetworkReply::InsecureRedirectError