[Solved] sending QByteArray from QNetworkAccessManager

[seang96]

I currently have two issues. The first one is that with my compareDownload function revised to the following code below, images are not saved because it turns null. The second issue is that in compareDownload the QByteArray does not seem to be what it should be and does not match readAll() of the file.

void NewsBulletin::Download(const QString url) //Starts downloading the desired url
{
    waitFor(1000+(download_que*100), &currently_downloading);
    QString extension;
    if(url.contains(".png"))
        extension = "png";
    else if(url.contains(".jpg"))
        extension = "jpg";
    else if(url.contains(".gif"))
        extension = "gif";
//    qDebug() << "The extension is: " << extension;
    current_url = url;
//    qDebug() << url;
    manager = new QNetworkAccessManager(this);
    if (extension == "png" || extension == "jpg" || extension == "gif")
    {
        image_extensions += extension;
        connect(manager, SIGNAL(finished(QNetworkReply*)),
                this, SLOT(imageReplyFinished(QNetworkReply*)));
    }
    else
        connect(manager, SIGNAL(finished(QNetworkReply*)),
                this, SLOT(siteReplyFinished(QNetworkReply*)));
    if (download_que != 0)
        download_que--;
    manager->get(QNetworkRequest(QUrl(url)));
    currently_downloading = true;
}

void NewsBulletin::siteReplyFinished (QNetworkReply *reply) //Function that saves site.html if it loads
{
    currently_downloading = false;
    QString filename = "site.html";
//    qDebug() << "This is reply before compareDownload from siteReply";
//    qDebug() << reply->readAll();
    QByteArray downloaded_content = reply->readAll();
    qDebug() << downloaded_content.size();
    QFile *dump = new QFile(application_path + "downloaded_content_dump.txt");
    if(dump->open(QFile::Append))
    {
        dump->write(downloaded_content);
        dump->close();
    }
    if(reply->error())
    {
        qDebug() << "ERROR!";
        qDebug() << reply->errorString();
    }
    else if(compareDownload(downloaded_content, filename/*, new_filename*/))
    {
        removeFile(application_path, filename);
        qDebug() << "My new compare worked!";
        QFile *file = new QFile(application_path + /*new_*/filename);
        if(file->open(QFile::Append))
        {
            file->write(downloaded_content);
            file->close();
        }
        delete file;
        QStringList site_html;
        qDebug() << "created QStringList site_html";
        pageToStrings(site_html, filename);
        qDebug() << "completed pageToStrings";
        parsePage(site_html, current_url);
        qDebug() << "completed parsePage";
    }
    reply->deleteLater();
}

bool NewsBulletin::compareDownload(QByteArray new_contents, QString filename)
{
    bool return_what = false;
    qDebug() << "I am in compareDownload";
//    qDebug() << new_contents[1];
//    qDebug() << new_contents[1] << endl
//             << new_contents[2];
    QFile file(application_path + filename);
    if (file.exists())
    {
//        QString new_contents_qstr(new_contents);
        file.open(QIODevice::ReadOnly | QIODevice::Text);
        QTextStream in(&file);
        QTextCodec::setCodecForLocale(QTextCodec::codecForName("UTF-8"));
        //QString file_contents = in.readAll();
        new_contents.replace("\n\n", "\n");
        QByteArray file_byte_array = in.readAll().toLocal8Bit();
        while (new_contents.contains("\n\n"))
        {
            new_contents.replace("\n\n", "\n");
        }
        new_contents.replace("^M", "");
        while (file_byte_array.contains("\n\n"))
        {
            file_byte_array.replace("\n\n", "\n");
        }
        file.close();
        qDebug() << "outputting new file array";
        qDebug() << new_contents[5] << new_contents.size();
        QFile *dump = new QFile(application_path + "new_contents_dump.txt");
        if(dump->open(QFile::Append))
        {
            dump->write(new_contents);
            dump->close();
        }
        qDebug() << "outputting old file array";
        qDebug() << file_byte_array[5] << file_byte_array.size();
        QFile *dump_2 = new QFile(application_path + "file_byte_array_dump.txt");
        if(dump_2->open(QFile::Append))
        {
            dump_2->write(file_byte_array);
            dump_2->close();
        }
        for (int i=0; i<=file_byte_array.size(); i++)
        {
            if (file_byte_array[i] != new_contents[i])
            {
                return_what = true;
                break;
            }
            else if (i == file_byte_array.size())
            {
                qDebug() << "compareDownload will return false, duplicate file.";
                return_what = false;
            }
        }
    }
    else
    {
        qDebug() << "compareDownload will return true, DNE.";
        return_what = true;
    }
    return return_what;
}

Edit: updated the code, fixed the first issue. I am starting to think that the issue in compareDownload is incorrect encoding. Before I was comparing a QString to the QByteArray, but with the updated code, I am comparing QString to QString, however the converted value does not seem correct. From what I see, in order to do what i want I need to convert from QByteArray to Qstring correctly, or convert the in.readAll() QString to QByteArray.
Edit2: Updated code again.
Edit3: After reading the api for hours, I found the reason for the bytes being different.
file.open(QIODevice::ReadOnly | QIODevice::Text);
QIODevice::Text needs to be removed. This flag changes end of line terminators into the terminator for cpp, "\n", thus giving a byte difference.

[seang96]

I figured out what is going on, sort of. download_content in QByteArray seems to contains ^M (MSDOS newline) in some parts of the file when written to dump. When written to the desired file, it does not have those. Later on, in compareDownload the QBytearray does ontain the "^M" characters and the written file does not, therefore their bytes are different by x*^M. I currently do not understand why ^M is there and then not when written and I do not know how to remove from the QByteArray in the compareDownload function. Also during my tests to figure this out, QByteArray new_contents saves to file with ^M once I added that in there to compare with QByteArray file_byte_array.
Edit: Actually the regular file that becomes file_byte_array does have the ^M, but the QByteArray that reads the file lacks it.