How to get the whole file from TCP server?

kayakaan02

So I asked a question yesterday about how can I access what I receive from TCP server in this(https://doc.qt.io/qt-6/qtscxml-ftpclient-example.html) example of project.

But QByteArray's size is not enough for the whole file.

So I want to access the file directly and save it.

If it is not possible, then how do I solve the size problem of the QByteArray? maybe on a pointer array but is there a more known way to use in this kind of situations?

mpergand

Create and open a QFile for writting before you start the download,
and write data to the file as they arrive.

QObject::connect(&dataChannel, &FtpDataChannel::dataReceived,
                     [](const QByteArray &data) {
       myFile->write(data);
    });

See docs about QFile.

kayakaan02

@mpergand Thank you, but this doesn't resolve the problem.

I think QByteArray's size is too small, for example the file is 1450kb but it can only write first 64kb of it

kayakaan02

@mpergand So I first opened the file before receiving data, to empty the file. And then wrote the data to file in append mode. But the size is not consistent?

Sometimes it's 320kb sometimes 250kb sometimes 550kb and so on.

Any idea what can be causing this or how can I solve it?

jsulm

@kayakaan02 Please show how exactly you open the file.

kayakaan02

This is how I open the file, because I think the connect is repeating itself, I first opened the file and emptied it with WriteOnly and empty string, and then appended the data. b.txt is 1401 kb and test.txt changes(350-680kb).

QString server = "127.0.0.1";
QString file = "b.txt";

QCoreApplication app(argc, argv);
FtpClient ftpClient;
FtpDataChannel dataChannel;
FtpControlChannel controlChannel;

QFile myfqle("C:\\test.txt");
myfqle.open(QIODevice::WriteOnly);
myfqle.write("");
// Print all data retrieved from the server on the console.
QObject::connect(&dataChannel, &FtpDataChannel::dataReceived,
                 [](const QByteArray &data) {
    QFile myfqle("C:\\test.txt");
    myfqle.open(QIODevice::Append);
    myfqle.write(data);
    });

edit: I also tried myfqle.write(data.constData()); it expands the range a few hundred kbs but still not enough

jsulm

@kayakaan02 said in How to get the whole file from TCP server?:

QFile myfqle("C:\test.txt");

You should not open the file every time you get new data! Opening an closing files is a time consuming operation.
Did you also check whether the whole file was actually transferred?

kayakaan02

@jsulm

QFile myfqle("C:\\test.txt");
myfqle.open(QIODevice::WriteOnly);
myfqle.write("");
// Print all data retrieved from the server on the console.
QObject::connect(&dataChannel, &FtpDataChannel::dataReceived,
                 [&myfqle](const QByteArray &data) {
    myfqle.write(data.constData());
    });

This gives me better results but still not enough. I can get data up to 1200-1300kb's consistently but it gets worse with the data size.

Any way I can optimize this more?

Also for the size of 4203kb it outputs a file in range of 2000-3000kb range.

kayakaan02

Edit: I solved the problem by using Sleep(1) at the end of every buffer uploading from ftp client.

kayakaan02

@jsulm I also have another question.

So when transmitting a rar file, with using data.constData() it cannot create the rar file. But it has no problem with only using data.

Is constData() really important/what does it really do?

JonB

@kayakaan02 said in How to get the whole file from TCP server?:

Edit: I solved the problem by using Sleep(1) at the end of every buffer uploading from ftp client.

You really ought not and should not need to do any such thing. I cannot believe writing to a file cannot keep up with FTP reception.

The first thing to note is I do not see where you close() the file? Until you close a file you are writing to you may get an "incorrect" size.

So when transmitting a rar file, with using data.constData() it cannot create the rar file. But it has no problem with only using data.

Again something fishy here! Both should work equally.

Another question: what is the scope/lifetime of the QFile myfqle you show? Is it a local variable in some function? Does it definitely live until all the bytes have been received, or could it go out of scope before it has finished receiving?

kayakaan02

It is the exact code from the example "Qt SCXML FTP Client Example". It will be a bit long but let me put the main.cpp:

struct Command {
    QString cmd;
    QString args;
};

int main(int argc, char *argv[])
{
//    if (argc != 3) {
//        qDebug() << "Usage: ftpclient <server> <file>";
//        return 1;
//    }

    //QString server = QString::fromLocal8Bit(argv[1]);
   // QString file = QString::fromLocal8Bit(argv[2]);
    QString server = "127.0.0.1";
    QString file = "b.txt";

    QCoreApplication app(argc, argv);
    FtpClient ftpClient;
    FtpDataChannel dataChannel;
    FtpControlChannel controlChannel;

///////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////
    //HERE///////////////////////////////////////////////////////////////////////////////////
    QFile myfqle("C:\\Users\\thevo\\Desktop\\aaf5h6j\\test7.txt");
    myfqle.open(QIODevice::WriteOnly);
    // Print all data retrieved from the server on the console.
    QObject::connect(&dataChannel, &FtpDataChannel::dataReceived,
                     [&myfqle](const QByteArray &data) {
        myfqle.write(data);
    });
///////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////////////


    // Translate server replies into state machine events.
    QObject::connect(&controlChannel, &FtpControlChannel::reply, &ftpClient,
                     [&ftpClient](int code, const QString &parameters) {
        ftpClient.submitEvent(QString("reply.%1xx")
                              .arg(code / 100), parameters);
    });

    // Translate commands from the state machine into FTP control messages.
    ftpClient.connectToEvent("submit.cmd", &controlChannel,
                             [&controlChannel](const QScxmlEvent &event) {
        controlChannel.command(event.name().mid(11).toUtf8(),
                               event.data().toMap()["params"].toByteArray());
    });

    // Commands to be sent
    QList<Command> commands({
        {"cmd.USER", "admin"},// login
        {"cmd.PORT", "21"},         // announce port for data connection,
                                  // args added below.
        {"cmd.RETR", file}        // retrieve a file
    });

    // When entering "B" state, send the next command.
    ftpClient.connectToState("B", QScxmlStateMachine::onEntry([&]() {
        if (commands.isEmpty()) {
            app.quit();
            return;
        }
        Command command = commands.takeFirst();
        qDebug() << "Posting command" << command.cmd << command.args;
        ftpClient.submitEvent(command.cmd, command.args);
    }));

    // If the server asks for a password, send one.
    ftpClient.connectToState("P", QScxmlStateMachine::onEntry([&ftpClient]() {
        qDebug() << "Sending password";
        ftpClient.submitEvent("cmd.PASS", "qt");
    }));
    qDebug() << "hi" << "\n";
    QString hi;

    // Connect to our own local FTP server
    controlChannel.connectToServer(server);
    QObject::connect(&controlChannel, &FtpControlChannel::opened,
                     [&](const QHostAddress &address, int) {
        dataChannel.listen(address);
        hi= dataChannel.portspec();
        qDebug() << "hi" << hi<< commands[1].args << "\n";
        commands[1].args = selam;

        ftpClient.start();
    });

    qDebug() << "file: " << file << "\n";


    return app.exec();
}

I put the part inside of a lot of "/"'s so it's easier to notice.

If I close the file in or after the QObject::connect scope it just says device not open and no data in file at all so I assume it runs main more than once to get the data.

I tried to declare myfqle outside of the main scope but it generated some errors.

Also at the end of the main I did some extra changes just to see how things work but that doesn't make any difference

It acts weird at constData() with rar files that makes me wonder why :D

JonB

@kayakaan02
FWIW this code never sees completion of the file transfer and closes QFile myfqle. Are you looking at the file content/size while the program is still running or after you have exited it? Because the former will not give reliable size/content.