how to send/receive mutiple files with qtcpsocket

[SpartaWHY117]

for now i have four files to send,i have writen following send code,is it correct?

	QByteArray f1 = powerFile.readAll();
	QByteArray f2 = cpuFile.readAll();
	QByteArray f3 = memFile.readAll();
	QByteArray f4 = diskFile.readAll();
        QByteArray block1;
	QByteArray block2;
	QByteArray block3;
	QByteArray block4;

	QDataStream out1(&block1, QIODevice::WriteOnly);
	QDataStream out2(&block1, QIODevice::WriteOnly);
	QDataStream out3(&block1, QIODevice::WriteOnly);
	QDataStream out4(&block1, QIODevice::WriteOnly);

	out1.setVersion(QDataStream::Qt_5_7);
	out2.setVersion(QDataStream::Qt_5_7);
	out3.setVersion(QDataStream::Qt_5_7);
	out4.setVersion(QDataStream::Qt_5_7);


	out1 << static_cast<qint32>(TransmissionType::rrdToolData);
	out2 << static_cast<qint32>(TransmissionType::rrdToolData);
	out3 << static_cast<qint32>(TransmissionType::rrdToolData);
	out4 << static_cast<qint32>(TransmissionType::rrdToolData);

	

	out1 << g_totalPower_RRD;
	block1.append(f1);
	out2 << g_Cpu_RRD;
	block2.append(f2);
	out3 << g_Mem_RRD;
	block3.append(f3);
	out4 << g_Disk_RRD;
	block4.append(f4);

	qint64 x = 0;
	while (x < block1.size()) {
		qint64 y = tcpSocket->write(block1);
		x += y;
	}
	x = 0;
	while (x < block2.size()) {
		qint64 y = tcpSocket->write(block2);
		x += y;
	}
	x = 0;
	while (x < block3.size()) {
		qint64 y = tcpSocket->write(block3);
		x += y;
	}
	x = 0;
	while (x < block4.size()) {
		qint64 y = tcpSocket->write(block4);
		x += y;
	}

for receive those files,just according to their file name to decide write in which file. Is this the workable ideal?

[VRonin]

there is no need to use temporary QByteArray if the files are big you are occupying a lot of memory this way

QFile* const fileArray[] = {&powerFile,&cpuFile,&memFile,&diskFile};
const decltype(g_totalPower_RRD)* const headerArray[]={&g_totalPower_RRD,&g_Cpu_RRD,&g_Mem_RRD,&g_Disk_RRD};
QDataStream out(tcpSocket);
out.setVersion(QDataStream::Qt_5_7);
char buffer[1024*512]; // 0.5 MB
static_assert(std::extent<decltype(fileArray)>::value == std::extent<decltype(headerArray)>::value, "Array size mismatch");
for(int i=0;i<std::extent<decltype(fileArray)>::value;++i){
    out 
            << static_cast<qint32>(TransmissionType::rrdToolData)
            << *(headerArray[i]);
    Q_ASSERT(fileArray[i]->isOpen());
    Q_ASSERT(fileArray[i]->isReadable());
    for(;;){
        const qint64 readBytes= fileArray[i]->read(buffer,std::extent<decltype(buffer)>::value);
        if(readBytes<=0)
            break;
        if(out.device()->write(buffer,readBytes)<0)
            break;
    }
}

[stryga42]

said in how to send/receive mutiple files with qtcpsocket:

qint64 x = 0;
while (x < block1.size()) {
qint64 y = tcpSocket->write(block1);
x += y;
}

will only work in the straight forward case when write takes the full size of your block at once. If it fails, it will return -1 and you loop endlessly. If it takes only part of your block the second iteration of the loop will pass the same buffer again and send again the start of you file. block1 is a QByteArray, not a stream which "consumes" bytes as it is read.

[SpartaWHY117]

@VRonin thanks for your replay.
compiler says const file can't match the function read, and what the meaning of code std::extent<decltype(buffer)>::value?

what's more , if use the fixed size char[] to read the file if the file size large than the 0.5MB,in your code the file will write more than once into the socket,so it will send two blocks of the file ?then how to receive it?

[VRonin]

@SpartaWHY117 said in how to send/receive mutiple files with qtcpsocket:

compiler says const file can't match the function read

My bad, const QFile* const sould just be QFile* const, changed in the original post now

what the meaning of code std::extent<decltype(buffer)>::value

The compiler translates it to 524288 (1024*512) it's just so you can change the size of the buffer in the declaration and it automatically gets picked up in the read

what's more , if use the fixed size char[] to read the file if the file size large than the 0.5MB,in your code the file will write more than once into the socket,so it will send two blocks of the file ?then how to receive it?

that does not impact how the file is read or written. what that says is basically do not load more than 0.5MB in RAM at a time, if the file is bigger it will just write the first 0.5MB chunk, then the next and so on

[SpartaWHY117]

@VRonin
thanks again.I get it now. what's more ,I have not said clearly before,the valueg_totalPower_RRD and etc is the name of file.
And i still don't understand code const decltype(g_totalPower_RRD)* const headerArray[],why decltype(g_totalPower_RRD)need added? and as you say the fixed size char just control the mem usage for porgram,the above code write four files into the one block?

for now ,i am misunderstanding the tcp and udp,does tcp has the concept of blocks?or tcp is just the stream? so if use your fixed size send code, how to write the correct receive code?how to read the input stream to each file? is the following code workable?

QString  filename;
QDataStream in(socket);
in>>filename;
//need to determine read size of each file?
in>>file;

[VRonin]

@SpartaWHY117 said in how to send/receive mutiple files with qtcpsocket:

why decltype(g_totalPower_RRD)need added?

Because I did not know the type of g_totalPower_RRD, there is no mention of it in the original post, so I let the compiler to guess it

//need to determine read size of each file?

you are correct indeed, just add it after the header:

out 
            << static_cast<qint32>(TransmissionType::rrdToolData)
            << *(headerArray[i])
            << static_cast<qint64>(fileArray[i]->size())
;

move the Q_ASSERT above that line, just to be safe

also reading from the socket is not as straightforward as the code you posted because it's possible, for example, that only part of the filename reached the other end and you are trying to read it all.

see ClientSocket::readClient() on page 366 of this book for an example of how to do it

[SpartaWHY117]

@VRonin thanks your suggestions,according to your advice my current code is following
sever send a message that call client to send those xxx.rrd files to server.

the client sending key code

 QFile* const fileArray[] = { &powerFile,&cpuFile,&memFile,&diskFile };
	 QStringList fileNameList;
	 fileNameList<< g_totalPower_RRD << g_Cpu_RRD << g_Mem_RRD << g_Disk_RRD;
	QDataStream out(tcpSocket);
	out.setVersion(QDataStream::Qt_5_7);
	char buffer[1024 * 512]; // 0.5 MB
	
	for (int i = 0; i<std::extent<decltype(fileArray)>::value; ++i) {
		out
			<< static_cast<qint32>(TransmissionType::rrdToolData)
			<<fileNameList.at(i)
		    << static_cast<qint64>(fileArray[i]->size());
		for (;;) {
			const qint64 readBytes = fileArray[i]->read(buffer, std::extent<decltype(buffer)>::value);
			if (readBytes <= 0)
				break;
			if (out.device()->write(buffer, readBytes)<0)
				break;
		}
	}

the server receive slots key code

QTcpSocket *socket = qobject_cast<QTcpSocket *>(socketObject);
	qintptr id = socket->socketDescriptor();
	if (!socket || !socket->bytesAvailable())
		return;
	QDataStream in;
	in.setDevice(socket);
	in.setVersion(QDataStream::Qt_5_7);
	qint32 requestType=0;

	in.startTransaction();
	in >> requestType;
switch (requestType)
	{
	case TransmissionType::clientConfig:
	      {      //other code 
                        break;
                }
         case TransmissionType::rrdToolData:
	{
		QString fileName;
		in >> fileName;
		
		QFile newFile(fileName);
		newFile.open(QIODevice::WriteOnly);

		if(socket->bytesAvailable()<sizeof(quint64))
			return;
		in >> nextBlockSize;
		if (socket->bytesAvailable() < nextBlockSize)
			return;
		QByteArray filearry;
		filearry.resize(nextBlockSize);
		in.readRawData(filearry.data(), nextBlockSize);

		newFile.write(filearry);
		newFile.close();
		nextBlockSize = 0;
		break;
	} 
default:
	{
		qDebug() << "wrong type info"<< requestType;
			
	}
             ...

if (!in.commitTransaction())
		return;
    

however when i run the test,the socket will report the error:

QIODevice::startTransaction(qtcpsocket) :called while the transacation alreadly in progress
wrong type info 1381123072
.....

then I debug the code,i found the function will return in this code

in >> nextBlockSize;
if (socket->bytesAvailable() < nextBlockSize)
	return;  <------------- return at this postion

so how can i handle this problem ?actually because the function startTransaction introduced in latest version 5.7,so i didn't know how to use it correctly.

[VRonin]

you are trying to read data when the connection is enstablished but that's not how QTcpSocket works, it emits readyRead() whenever you can read something. Have a look at the sub-chapter about TCP data in the book linked above

[SpartaWHY117]

@VRonin
thanks again.I connected that function to readRead slot,that is correct. After check from the git history version compare,i found the problem:
what really embarrassing is that I didn't coordinate well serialized objects in client and sever ,some wrong happened in above other code
finally,thanks for your help anyway