Reading a file with Qt

EL-jos

Hello everyone

as the title indicates, I'm looking to read an entire file with Qt.

when I read a small size file it is not a problem but when I try to read for example a large size file that there is a problem, my problem is that the package(QByteArray) is not displayed in the console(so it is always empty), here is my code:

#include <QApplication>
#include <QFileDialog>
#include <QFile>
#include <QDebug>
 
int main(int argc,char** argv)
{
    QApplication app(argc,argv);
 
    // Path to the target file to copy
    QString chemin = QFileDialog::getOpenFileName(nullptr,"Path to the file");
 
    // We have our file that we pass as a parameter the path to our target file
    qDebug() << chemin;
    QFile fichier(chemin);
 
    fichier.open(QIODevice::ReadOnly);
    if(fichier.isOpen()){
 
        qDebug() << "File opened and here is the name: " << fichier.fileName();
 
        QFileInfo info(fichier);
        QString extension = info.suffix();
 
        QByteArray paquet;
 
        while(!fichier.atEnd()){
 
            paquet = fichier.readAll();// full reading of our byte file
        }
 
        if(!paquet.isEmpty()){
 
           qDebug() << "The data has been well serialized in the byte packet.";
           qDebug() << paquet;
        }
    }
    return app.exec();
}

Note: I will read an.txt file of 3.0MB knowing that it is a file that contains genre words from a dictionary.

in fact if I Serialize and deserialize my dictionary it is because I have to send it into a network,

so the serialization will be during sending to the network and the de-serialization will be at the destination.

Thanks in advance for your help.

mrjj

Hi
Tried you code directly
with
https://norvig.com/big.txt
which is 6MB file
and it just read it in.

"E:/big.txt"
File opened and here is the name: "E:/big.txt"
The data has been well serialized in the byte packet. 6488666

so are u sure your text file is just 3 MB ?

EL-jos

@mrjj said in Reading a file with Qt:

Hi
Tried you code directly
with
https://norvig.com/big.txt
which is 6MB file
and it just read it in.
"E:/big.txt"
File opened and here is the name: "E:/big.txt"
The data has been well serialized in the byte packet. 6488666
so are u sure your text file is just 3 MB ?

Of course, yes, just 3MB.

then your result has an error because the package must display all the contents of the file in the console but at home your result is :
The data has been well serialized in the byte packet. 6488666

SamurayH

Hi @EL-jos,
Be sure that your paquet contains the file's content, the problem could be with qDebug or the QtCreator's console.If you want to check that out you can use qlabel or qtextBrowser (it will consume some cpu for larger files), or QByteArray::mid.

QLabel *label = new QLabel(paquet);
label->showMaximized();
//--------------------------
qDebug() << paquet.mid(0,100);

I don't know if it's a length limitation by qDebug, but I don't think so since it isn't mentioned in the documentation.
You're not the first one, check out that thread to, I took the second line of code from it.

mchinand

What is the size of paquet after you have read from your file? Does it match the size of the file in bytes? qDebug() has a limited buffer that is probably the issue in this case. Is there other processing of this data that you do after reading it in? Does that still work? Also, if you are reading in a text file, you might want to use QTextStream instead of QByteArray.

Christian Ehrlicher

@EL-jos said in Reading a file with Qt:

    while(!fichier.atEnd()){

        paquet = fichier.readAll();// full reading of our byte file
    }

When here readAll() does not return the complete content with the first call, you will not read all... you want paquet += ...

EL-jos

Thanks for your help because I just noticed that my file was serialized in the QByteArray except the console could not display it in its entirety because of its buffer.

But there is a new problem: when I try to send my QByteArray that contains my serialize file, when the QByteArray arrives at the server it loses all its data so its bytes.

Please how to fix this problem?
Knowing that I use the TCP protocol.

JonB

@EL-jos
From the information you've supplied on the new problem, I would guess some code is wrong somewhere.

EL-jos

Here's my code on the client side:

void Controleur::on_pushButton_clicked()
{
    QString chemin = QFileDialog::getOpenFileName(this,"Chemin du fichier");
    cheminFichier->setText(chemin);

    QFile fichier(chemin);
    QFileInfo info(fichier);
    QString extension = info.suffix();

    if(fichier.exists()){

       qDebug() <<"LE fichier existe bel est bienet et voici son extension: " << extension;

       int reponse = QMessageBox::question(this,"Confirmation d'envie du fichier", "êtes-vous sûr d'<strong>envoyer</strong> le fichier: " + fichier.fileName(), QMessageBox::Yes | QMessageBox::No);
       if(reponse == QMessageBox::Yes){

           qDebug("Donc vous souhaitez envoyer le fichier dans le réseau.");

           fichier.open(QIODevice::ReadOnly);
           if(fichier.isOpen()){

               qDebug() << "Fichier ouvert et voici le nom: " << fichier.fileName();

               QByteArray paquetFichier;

               while(!fichier.atEnd()){

                   paquetFichier = fichier.readAll();// lecture entier de notre fichier en octet
               }

               fichier.close();

               if(!paquetFichier.isEmpty()){

                  qDebug() << "Les données ont été bien sérialiser dans le paquet en octets.";
                  qDebug() << paquetFichier;
               }


               QByteArray paquetEnvoieFichier;
               QDataStream outFichier(&paquetEnvoieFichier,QIODevice::WriteOnly);

               outFichier << (quint16) 0;
               outFichier << (quint16) 6;
               outFichier << extension;
               outFichier << paquetFichier;
               outFichier.device()->seek(0);
               outFichier << (quint16) (paquetEnvoieFichier.size() - sizeof(quint16));
               m_client->getSocket()->write(paquetEnvoieFichier);
           }// Fin fichier isOpen
          } // Fin MessageBox::YES
       } // Fin si le fichier existe
}

and this is the one for reading

if(id == (quint16) 6){

        qDebug() << "réception d'un fichier venant du serveur.";

        QString extension;
        QByteArray  paquetReception;

        in >> extension;
        in >> paquetReception;

        qDebug() << "voici l'extension du fichier: " << extension <<" le paquet est: " <<paquetReception;

        QFile fichier("AutreFichier."+ extension);
        fichier.open(QIODevice::WriteOnly);

        if(fichier.isOpen()){

            qDebug("Le fichier de la copie se -t- ouvert en écriture seul.");
            qDebug() << "avec l'extension: " + extension;

            QDataStream outAutreFichier(&fichier);
            outAutreFichier << paquetReception;
        }

Christian Ehrlicher

My comment is still not fixed... and now - what do you think will happen when your file size is greater than 65535 bytes?

outFichier << (quint16) (paquetEnvoieFichier.size() - sizeof(quint16));
And then you don't write the data directly to a file but stream it as QDataStream - this will add some more information in the front which is definitly not what you want... use
QFile::write(QByteArray())

And when sending data via network, readyRead() will for sure not contain the whole content - at least not when you're sending 6MB. Please take a look at the QTcpServer/Client examples (and avoid using QDataStream here - it's not needed at all...)

EL-jos

Yes I have this problem because I noticed that if my file size is larger than 65535 bytes, the server does not receive all the information. But how do we solve this problem?

Then if I use Qdatastream this to first write the identifier of my message because the one that says if it is a text message (5), if it is a new connection (3), if it is a disconnect (4) and for the file the identifier value is (7) to tell the server that it is a file. Here the QDATASTREAM allows me to write a screen like this:

*Message size

*Identifier

*Message

So if I don't use a Qdatastream how can I write in my QByteArray?

Translated with www.DeepL.com/Translator

aha_1980

Hi @EL-jos,

What do you want to do exactly?

For file transfer, decicated protocols exists, e.g. FTP or HTTP. To me it looks like your are re-inventing the wheel?

Regards

EL-jos

Concretely I want my client to send a file to my server and the server after receiving the file, that it sends this file to all the client that are connected to it.
Because it is a chat software then I also want the client to be able to send a file to another client via a server.

Christian Ehrlicher

Please take a look at the examples (again...) how to send large data ... http://doc.qt.io/qt-5/qtnetwork-loopback-example.html

EL-jos

Okay Thanks, I'll read this example and then I'll come back to you to give you the follow-up.

Thank you

EL-jos

Hi,
after several studies of the example given by the Qt documentation, I don't know where and how to send my file to the server?

then in the example given by the Qt documentation, just sends the same 64KB packet several times until it tints 50MB . and the server and the client are in the same project but in my case the client and the server are different projects

SGaist

Hi,

Are you writing both the client and server parts ?

EL-jos

Yes, do you want me to mail them?

SGaist

It was regarding your last remark about the server and client being in the same project. Since you are writing both, then it doesn't matter that they are in different projects.

EL-jos

Hey, everybody,

Still in the example given by the Qt documentation for the exchange of large documents between the client and the server in a chatt software, I could separate the client and the server in different projects which is not the case in the example given by the Qt documentation.

So here's my question:

Just that there I do not know how to adapt this example to my chatt software to be able to send large files between client and server?

Translated with www.DeepL.com/Translator

SGaist

There's nothing special to do, if you implement the client and the server the same way as the exemple, then it's also going to work the same.

EL-jos

Theoretically yes, but in practice this is not the case.

mrjj

@EL-jos
Hi
As mr @Christian-Ehrlicher noted higher up.
You are using quint16 for size handling
Try
#include <limits.h>
qDebug() << std::numeric_limits<quint16>::max();

and you will see the magic 65535 as maximum value.

EL-jos

Yes you are right but except that 65535 makes a maximum size of 64Ko or me I want to send voluminous documents of the order of 30Mo to even 50Mo

mrjj

@EL-jos
then use quint64 instead.

JKSH

@EL-jos said in Reading a file with Qt:

65535 makes a maximum size of 64Ko or me I want to send voluminous documents of the order of 30Mo to even 50Mo

Then you cannot use quint16. It is too small.

Use quint32 or even quint64 instead.

EDIT: @mrjj beat me to it!

EL-jos

Hey, everybody.
I was able to solve my problem thanks to your advice, now I am able to send large document in my chatt software but there is a new problem that arises, here is the problem: Which class should I use to send a video file in my chatt software?
Example: for image exchange, I used the class QImage(Stream for image)but now what class used for Video(Stream video)?

jsulm

@EL-jos If it is a file then simply send the binary content of the file.
There is no need to use a dedicated class for each and every data type.
https://stackoverflow.com/questions/30288385/how-to-send-a-file-in-qt

EL-jos

Yes it is already what I do so I send data in bytes but except that it does not work because I have an error when reading my file like this file is not supported

aha_1980

@EL-jos

Why should a file not be supported if you just send it byte-by-byte? if you store it on a harddisk, it does not matter for the hard disk which file type it is. Its just a number of bytes.

Same happens when you send it over network.

Regards

jsulm

@EL-jos said in Reading a file with Qt:

error when reading my file like this file is not supported

Please explain. Where do you get this error? QFile (I guess you use it for reading) does not care at all about file content.

EL-jos

In fact after the client sends the byte array, the server receives the bytes and Opens a file in write only mode and writes all the bytes received but except that if I try via VLC to read this file, there is a message that appears like this file is not supported

jsulm

@EL-jos You should show how you receive the data (whole code not just a small part of it).

JonB

@EL-jos
So you need to do a little detective work to help yourself.

You have a file which is openable by "VLC" (whatever that is) at the client before you start, right? And you're saying after you have sent it to server it is not openable, right?

So start by using a regular file copy, outside of anything Qt, to copy the file from client to server. Make sure you keep the same filename extension. Then: Is the copied file openable on the server? Is it the same size as when you copy via Qt? Use a file content comparison tool (e.g. Windows fc /b), is the file content of the Qt copy identical to that of the non-Qt copy?

EL-jos

Okay I'll try to put your advice into practice then find my mistake if I still can't I'll have to post my code for you to see it