[Solved]QNetworkAccessManager sometimes does not download file

alamahant

@Christian-Ehrlicher
Thanks for the clarity.
How do I do what you are proposing.
In qDebug I see thousands of "incoming data" while the file is being downloaded.
Therefore it seems it is reading and writing in batches....
Please forgive my ignorance but i am not clear what you are proposing that i do......
:)

Christian Ehrlicher

@alamahant said in QNetworkAccessManager sometimes does not download file:

How do I do what you are proposing.

Don't understand - instead writing to a buffer in readyRead() write to you output file.

JonB

@alamahant
During your downloadBegin() (btw, bad name, this is attached to readyRead() so is called over & over) you are presently pushing all the bytes received into *mDataBuffer, and only writing the complete buffer to file at end in downloadEnd(). @Christian-Ehrlicher is pointing out that if the file is, say, 4GB long that will use up 4GB of memory --- bad idea. He wants you to open the output file at the start, and write the bytes received as they arrive in readyRead() to file at that point, so as not to use up memory.

alamahant

@all
Yes I tried the proposed solution but I still cant get certain files to download, while others -- equally large download just fine.
I think its a client/server negotiation issue but i am clueless how to debug it.
Should I maybe set download speed limit or something of the sort?
Or is it an SSL thing.
Does Qt use certificates and where are they located?
Or best how to force it to not close any connection on ssl errors if --indeed the closing happens by Qt.
I dont get any info also from my connect error statement and the pethod associated with it.
Please see above...
Sory for tiring you
Thanks a lot

JonB

@alamahant
I don't know, other than I would have expected some sort of error.

The most obvious question is what are these URLs? If you have one file that downloads and another that does not, are they at the same site/in the same directory, or in different sites, or what? Are these across https? Are you able to visit these URLs in a browser, outside of Qt application?

alamahant

For example This downloads fine
https://alpha.de.repo.voidlinux.org/live/20170220/void-live-x86_64-20170220-xfce.iso

BUT this closes the connection

https://cdimage.debian.org/cdimage/bullseye_di_alpha2/amd64/iso-cd/debian-bullseye-DI-alpha2-amd64-netinst.iso

It seems this is nOT a size amatter as the second is much smaller than the first.
Nor an http/https issue as both are https.
So the logical conclusion is some kind of negotiation tls handshake error.
Both download fine with wget.
BUT I just noticed something in the wget output.

The first one(the one that downloads fine gives this output

wget https://alpha.de.repo.voidlinux.org/live/20170220/void-live-x86_64-20170220-xfce.iso
--2020-09-30 19:19:58-- https://alpha.de.repo.voidlinux.org/live/20170220/void-live-x86_64-20170220-xfce.iso
Resolving alpha.de.repo.voidlinux.org... 95.216.76.97
Connecting to alpha.de.repo.voidlinux.org|95.216.76.97|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 558891008 (533M) [application/octet-stream]
Saving to: ‘void-live-x86_64-20170220-xfce.iso’

void-live-x86_64-20170220-xfc 0%[ ] 2.60M 1.45MB/s

WHEREAS the secon -- the one fails to download gives this:

wget https://cdimage.debian.org/cdimage/bullseye_di_alpha2/amd64/iso-cd/debian-bullseye-DI-alpha2-amd64-netinst.iso
--2020-09-30 19:20:20-- https://cdimage.debian.org/cdimage/bullseye_di_alpha2/amd64/iso-cd/debian-bullseye-DI-alpha2-amd64-netinst.iso
Resolving cdimage.debian.org... 194.71.11.165, 194.71.11.173, 2001:6b0:19::165, ...
Connecting to cdimage.debian.org|194.71.11.165|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://saimei.ftp.acc.umu.se/cdimage/bullseye_di_alpha2/amd64/iso-cd/debian-bullseye-DI-alpha2-amd64-netinst.iso [following]
--2020-09-30 19:20:22-- https://saimei.ftp.acc.umu.se/cdimage/bullseye_di_alpha2/amd64/iso-cd/debian-bullseye-DI-alpha2-amd64-netinst.iso
Resolving saimei.ftp.acc.umu.se... 194.71.11.138, 2001:6b0:19::138
Connecting to saimei.ftp.acc.umu.se|194.71.11.138|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 360710144 (344M) [application/x-iso9660-image]
Saving to: ‘debian-bullseye-DI-alpha2-amd64-netinst.iso’

      debian-bullseye-DI-   0%[                                                ] 792.00K   732KB/s               ^C

IS THERE a redirect of some sort involved in the second?
And if yes how to adjustt my Qt code to cover for this?

Thankssssssss
:)

alamahant

Here is another one that does NOT work

wget https://bouncer.gentoo.org/fetch/root/all/releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz
--2020-09-30 19:29:39-- https://bouncer.gentoo.org/fetch/root/all/releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz
Resolving bouncer.gentoo.org... 140.211.166.178, 2001:470:ea4a:1:225:90ff:fe02:1957
Connecting to bouncer.gentoo.org|140.211.166.178|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://mirror.isoc.org.il/pub/gentoo//releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz [following]
--2020-09-30 19:29:41-- https://mirror.isoc.org.il/pub/gentoo//releases/amd64/autobuilds/20200923T214503Z/stage3-amd64-20200923T214503Z.tar.xz
Resolving mirror.isoc.org.il... 192.115.211.70, 2a01:4280:2:211a::70
Connecting to mirror.isoc.org.il|192.115.211.70|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 219663964 (209M) [application/octet-stream]
Saving to: ‘stage3-amd64-20200923T214503Z.tar.xz’

         stage3-amd64-202   0%[                                                ]  39.40K  17.8KB/s               ^C

What all the failed ones have in common is that there is some short of redirect involved....
Or is it Apache vs Nginx?
Where should I look to solve this?
Thanks Guys
:)

Christian Ehrlicher

@alamahant said in QNetworkAccessManager sometimes does not download file:

short of redirect involved....

correct, see https://doc.qt.io/qt-5/qnetworkrequest.html#Attribute-enum / QNetworkRequest::RedirectionTargetAttribute

alamahant

@Christian-Ehrlicher
Yessssssssssssssss.
You are a life saver.
After setting
request.setAttribute(QNetworkRequest::FollowRedirectsAttribute, 21);

It WORKS!!!!!
Thanks a lot....
:)

Christian Ehrlicher

@alamahant Nice, hope you leave the direct write to a file in your code since it saves a lot of memory :)