TCP connection for file transfer

aha_1980

@Chandras002 said in TCP connection for file transfer:

LAN environment has 10 computer's connected with Ethernet. One computer act as server the remaining act like client.

Then why reinvent the wheel? File sharing already exists. Keywords: CIFS, Samba, NFS.

I didn't have a deeper look at QFtp, but I thought it is only a client?

VRonin

You don't need to use FTP protocol, you can send files directly via TCP.

open the file for reading with QFile
send a qint64 that tells the receiving end how big the file should be
use QFile::read and QTcpSocket::write to send chunks of the file
use QFile::write and QTcpSocket::read on the receiving end to reconstruct the file on the other end

robcont_

@VRonin
What if the data that has to be sent is a big QImage? It would be great to use the transaction mechanism. Please, give a look to my topic here.

VRonin

@robcont_ said in TCP connection for file transfer:

What if the data that has to be sent is a big QImage?

1MB is not big. You can stream it direcly:

void VCameraServer::sendTcpData()
{
        QDataStream out(mTcpClient);
        out.setVersion(QDataStream::Qt_5_11);
        QImage image(filePath);
if(image.isNull()) 
return;
        out << image;
}

void VCameraClient::newTcpDataRead()
{   
    QDataStream in(mTcpSocket);
QImage  image;
    in.setVersion(QDataStream::Qt_5_11);
    in.startTransaction();
    in  >> image;
    if ( !in.commitTransaction() ) 
return;
    qDebug() << "Recv:" << image.size() << "bytes; img name:" << "; image format:" << image.format();   
    if (image.isNull()) qDebug() << "Recv  image is null!";
    else update();
}

robcont_

@VRonin
Thanks for the quick reply. I tried your code but is not working. The log is the same as what I wrote in my first post here.
I connected my newTcpDataRead() slots to the readyRead() signal of QTcpSocket. What is not clear to me is why the transactions mechanism doesn't work properly when transferring a QImage. It seems like is not able to rebuild the data received in chunks.

I'm new on Qt and I'm struggling on this for days.

JonB

@robcont_
Not sure what you mean. @VRonin's code looks right to me. Transactions don't care (or know) whether QImages or anything else is being sent.

I looked at the other question. I note you were using out.device()->seek(0);. I don't suppose that has consequences for the transactions? I didn't understand why you are doing that, why do you have to go back and put a size in instead of putting it in in the first place before the data?

robcont_

@JonB

@VRonin's code looks right to me. Transactions don't care (or know) whether QImages or anything else is being sent.

@VRonin's code is very close to what I wrote the first time (please take a look here). It works well with small data transferred (strings, numbers, lists) in a single chunk, but not with "bigger" QImage sent over TCP in many chunks. I followed the Fortune Client example , I read the documentation about QDataStream and transactions and I know it was built to simplified data streaming.
This is why I'm not understanding what's going wrong. Any suggestion or sample code is welcome.

I looked at the other question. I note you were using out.device()->seek(0);. I don't suppose that has consequences for the transactions? I didn't understand why you are doing that, why do you have to go back and put a size in instead of putting it in in the first place before the data?

Because when serializing a QImage the size of eachQByteArray block to be sent is not the same as the image itself. The client side must know exactly how many bytes to read, that is the block size not the image size. Below the log of my server part.

CameraServer: written 452690; block 452690; image 921600
CameraServer: written 551787; block 551787; image 921600
CameraServer: written 550869; block 550869; image 921600
CameraServer: written 550516; block 550516; image 921600
CameraServer: written 551006; block 551006; image 921600
CameraServer: written 550753; block 550753; image 921600
CameraServer: written 550669; block 550669; image 921600
CameraServer: written 550158; block 550158; image 921600
CameraServer: written 549714; block 549714; image 921600
CameraServer: written 549828; block 549828; image 921600
[...]

VRonin

@robcont_ said in TCP connection for file transfer:

Because when serializing a QImage the size of eachQByteArray block to be sent is not the same as the image itself. The client side must know exactly how many bytes to read, that is the block size not the image size. Below the log of my server part.

Datastream transactions already take care of this automatically, no need to do anything manually

robcont_

@VRonin said in TCP connection for file transfer:

Datastream transactions already take care of this automatically, no need to do anything manually

I know, but the code is not working properly and I'm not understanding why. This is why I asked for help on this forum.
As I said, when reading the stream with transactions (i.e your sample code) this is my log:

QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
QIODevice::ungetChar  (QTcpSocket): Called while transaction is in progress
Qimage is null!

VRonin

@robcont_ said in TCP connection for file transfer:

QIODevice::ungetChar (QTcpSocket): Called while transaction is in progress

do you have 2 threads operating on the same socket?

robcont_

@VRonin said in TCP connection for file transfer:

do you have 2 threads operating on the same socket?

No.
My VCameraClient is a custom QML components, inherits from QQuickPaintedItem and manages mTcpClient. When the readyRead() signal is fired up, then the newTcpDataRead() slot is triggered. Here datastream is read and the QQuickPaintedItem is repaint through the update() method.

JonB

@robcont_

Because when serializing a QImage the size of eachQByteArray block to be sent is not the same as the image itself. The client side must know exactly how many bytes to read, that is the block size not the image size.

Apart from @VRonin telling you that is not necessary, which I leave you to sort out with him/others. Even if you do need to do it your way, my issue is with your use of out.device()->seek(0); when transactions are involved. I do not know, but I'm wondering whether it might interfere with how they work. I am suggesting that to do it your way you should at least try not having to seek. Instead, serialize it to a buffer and get your size from there. Then build the block to be sent, as count followed by data, but do it left-to-right, no seeking. You should take this with a pinch of salt: if @VRonin says you need not be doing what you are, you should deal with that.

robcont_

@JonB said in TCP connection for file transfer:

Apart from @VRonin telling you that is not necessary, which I leave you to sort out with him/others.

I'm not arguing or questioning what @VRonin said. As I said in previous posts, I know that with transactions is not necessary to send the size of the block, in fact if you look at this post I did not.

Even if you do need to do it your way, my issue is with your use of out.device()->seek(0); when transactions are involved. I do not know, but I'm wondering whether it might interfere with how they work. I am suggesting that to do it your way you should at least try not having to seek. Instead, serialize it to a buffer and get your size from there. Then build the block to be sent, as count followed by data, but do it left-to-right, no seeking. You should take this with a pinch of salt: if @VRonin says you need not be doing what you are, you should deal with that.

I use out.device()->seek(0) in this last solution, which does NOT use transactions. Anyway, I'll try to follow your advice to buffer the stream, get the size from there and finally build the block.

Just for note, I tried @VRonin's code as is, but it gives the same output as mine (here).

JonB

@robcont_
I have a tiny/question thought for you (just guessing, I have no evidence, it might be nonsense). From your code:

        // serialize
        out << qint64(0) << image;
        out.device()->seek(0);
        out << (qint64)(block.size() - sizeof(qint64));  // the size of the block to be sent

You subtract sizeof(qint64) to allow for your original out << qint64(0), right? But how do you know that when serialized a qint64 occupies sizeof(qint64) bytes? Maybe it does, I don't know, but maybe there is some extra serialization info stored there too??

Another thought is: forget about your QImage. Practice sending a block of binary data, about the same size as an image, filled with 0, 1, 2, 3, ... 255, 0, 1, ... byte values, and verify you receive exactly that at the other side too? Because whatever your issue is, it should have nothing to do with being a QImage...

VRonin

@JonB said in TCP connection for file transfer:

Maybe it does

It does, but for a more generic case you can use:

const qint64 originalPos = out.device()->pos();
out << qint64(0);
const qint64 sizeOfHeader = out.device()->pos() - originalPos;
out << image;
out.device()->seek(0);
out << static_cast<qint64>(block.size() - sizeOfHeader );