Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

How to download an uncompressed (raw) file with QNetworkAccessManager?



  • On 5.9.1, QNetworkAccessManager uncompress automatically when he receives a gzip file.
    How can I prevent this?

    In fact, he fails to decompress concatenated gzip files. So I want to download the raw file and use a third-party to decompress it.

    I tried setting "Accept-Encoding" to "gzip" or to "" (in QNetworkRequest.setRawHeader) but it does nothing and I still get the decompressed data.





  • As I said, I already tried to set accept-encoding and it doesn't work.
    Posts were at 2011 so it's a bit old ;) and maybe no more right.
    I'm on Qt 5.9.1


  • Lifetime Qt Champion

    Hi,

    Can you provide a minimal compilable examples that allows to test that ?



  • Here is an abstract:

    
    class MyClass : public QNetworkAccessManager
    {
    	Q_OBJECT
    public:
    	MyClass(){}
    public slots:
    	void MyClass::get(const QString& p_urlRequest) 
    	{
    		QNetworkRequest networkRequest(p_urlRequest);
    		QNetworkReply * reply;
    		networkRequest.setRawHeader("Accept-Encoding", "gzip");
    		reply = this->get(networkRequest);
    		if(reply != NULL)
    		{
    			QList<QSslError> errors;
    			errors.push_back(QSslError::UnableToGetLocalIssuerCertificate);
    			errors.push_back(QSslError::CertificateUntrusted);
    			errors.push_back(QSslError::UnableToVerifyFirstCertificate);
    			reply->ignoreSslErrors(errors);
    			connect(reply, SIGNAL(finished()),p_client, SLOT(downloaded()) );
    			connect(reply, SIGNAL(error(QNetworkReply::NetworkError)),this, SLOT(error(QNetworkReply::NetworkError)) );
    			connect(reply, SIGNAL(sslErrors(QList<QSslError>)),this, SLOT(sslErrors(QList<QSslError>)) );
    			connect(reply, SIGNAL(downloadProgress(qint64, qint64)),this, SLOT(updateProgress(qint64, qint64)));        
    			connect(reply, SIGNAL(finished()),this, SLOT(finished()) );
    		}
    	}
    
    	void MyClass::error(QNetworkReply::NetworkError p_error)
    	{
    		QVariant statusCode = static_cast<QNetworkReply*>( sender())->attribute( QNetworkRequest::HttpStatusCodeAttribute );
    		qDebug() << "StatusCode:" <<statusCode;
    	}
    	void MyClass::updateProgress(qint64, qint64)
    	{
    			QNetworkReply * reply = dynamic_cast<QNetworkReply*>(sender());
    			QByteArray b = reply->readAll(); // check b : need to be raw
    	}
    	void MyClass::sslErrors(const QList<QSslError> &p_errors)
    	{
    			qDebug() << "SSL ERROR : " << p_errors;
    			for(int i = 0 ; i < p_errors.size() ; ++i)
    			{
    				std::cout << p_errors[i].error() << " ";
    			}
    	}
    	void MyClass::finished()
    	{
    	    qDebug() << "finished";
    	}
    }
    

  • Lifetime Qt Champion

    Would be nice to also have an URL to test.



  • I'm not allowed to do that from my provider.
    But I can give headers of the response (maybe there is something that explain why I don't get all the compressed file ). The fact is that even firefox uncompress badly when entering the url (https) : The response is a concatenation of gzip file and by bad, I mean that firefox (and Qt) only uncompress the first part of the file.
    If I want to do what it is supposed to do, I have to use curl (curl <url> -k -o myfile.gzip) and uncompress it with 7-zip. (I can't give neither the file because of data protected)

    Set-Cookie: DSSAPI-COOKIE=R2485808843; path=/
    Cache-Control: no-cache
    Pragma: no-cache
    Content-Length: 689
    Content-Type: text/plain
    Content-Encoding: gzip
    Expires: -1
    Accept-Ranges: bytes
    Server: Microsoft-IIS/7.5
    X-Request-Execution-Correlation-Id: 570a5823-9bca-4240-92b2-27af954a9110
    X-App-Id: Custom.RestApi
    X-App-Version: 11.2.653.64
    Date: Fri, 24 Nov 2017 23:17:37 GMT
    

    So, that lead me to come back to my first question. How can I force Qt to not auto decompress?

    In addition, here is a note from the provider:
    "The issue is that most HTTP clients do not fully support concatenated gzip files. When your REST API application downloads the gzip output using an HTTP client, if that client also attempts to decompress the output at the same time, there is a significant chance that it will fail to fully download or fully decompress the output.
    Most reports deliver output as a gzip file. If a report is large, it delivers its output as several smaller gzip files concatenated into a single large gzip file.
    If your HTTP client does not support concatenated gzip files, you may not receive some of your output. At this time, most HTTP clients do not fully support concatenated gzip files."



  • I created a concatenated file but we have to test it on a server that send it with a "Content-Type: text/plain" and "Content-Encoding: gzip"

    Finally, setting "accept-encoding" works. I don't know why it didn't work before (maybe I didn't look at the right place... my bad). But the issue of being not fully decompress is still here ^^


Log in to reply