send sqlite database file trought tcp

gfxx

I know is difficult but need to copy entire sqlite db trought tcp .... not understand reading online post the best approch ... for sure I understand that using BEGIN IMMEDIATE can pause db connection so can freely copy db on Qfile and send it ..... problem seems packet .... so there are a way to not corrupt sqlite file during tranfer via tcp?

regards

jsulm

@gfxx said in send sqlite database file trought tcp:

so there are a way to not corrupt sqlite file during tranfer via tcp?

Close all db connections, read the db file and send its content over TCP/IP connection.
"problem seems packet" - don't know what this means.

gfxx

In the past Seems my problem was solved .... but now I reuse the same call in other app an the result of qtcp transfer file become the same junk file trasmission ... for better understand it start in these manner:

"\x00\x00\x00\x00\x00\x14\xA0\x04\x00\x14\xA0\x00SQLite format 3\x00\x10\x00\x01\x01\x00@  \x00\x00G[\x00\x00\x01J\x00\x00\x00\x05\x00\x00\x01""2\x00\x00\x02\xD1\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00G[\x00.G\xA8\x05\x00\x00\x00\x01\x0F\xFA\x00\x00\x00\x00""c\x0F\xFA\x0F\xEE\x0F\xEE\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x

and that sequence of \x00 continue but everytime there are some redable string that contain database column description or table description .... plus database file in origin was 1.3Mb but at end of transmission obtain only 50kb (probabily last packet).

In the past I use this code for send:

    Database.removeDatabase("/home/de/.DB/mydb.sqlite3");

    if(tcpSocket->state() == QAbstractSocket::ConnectedState){
        QFile file("/home/de/.DB/mydb.sqlite3");
        if (!file.open(QIODevice::ReadOnly)){
            qDebug() << "file not open .....";
        }
        else{
                if(tcpSocket->state() == QAbstractSocket::ConnectedState){
                QByteArray block;
                while(1){
                    block.clear();
                    block = file.read(32768*8);
                    if(block.size() == 0) break;
                    tcpSocket->write(block);
                    tcpSocket->waitForBytesWritten();
                    block.clear();
                }
                 file.close();
            }

code for receive:

qint64 fileSize = client->bytesAvailable();
else if (fileSize > 200 && !fileBR)
        {
            QFile filew("/home/gr/.DB/receivedb.sqlite3");
            filew.resize(0);
            if(!filew.open(QIODevice::Append))
            {
                qDebug() << "....file not open  ....";
            }
            else{
                QByteArray read;
                //qDebug() << "Read    : " << read;
                while(client->bytesAvailable()){
                    read = client->read(client->bytesAvailable());
                    filew.write(read);
                    qDebug() << "Read this data   : " << read;
                }
             filew.close();

but these time I obtain these junk file instead the original copy ..... i try with buffer with same result .... problem seems to become from sender ....

Christian Ehrlicher

Your tcp socket handling is a mess - esp. on the receiver side since client->bytesAvailable() can be 0 even tough not all bytes were received. Use proper signal/slots, open the file and hold it as member and send the file size in the first 8 bytes so you know when you received all data.

gfxx

@Christian-Ehrlicher said in send sqlite database file trought tcp:

Your tcp socket handling is a mess - esp. on the receiver side since client->bytesAvailable() can be 0 even tough not all bytes were received. Use proper signal/slots, open the file and hold it as member and send the file size in the first 8 bytes so you know when you received all data.

ok .... so means if LAN was really good can work, but if "normal" impossible to obtain good result ... right? ....

any how really thans ...

I'm teaching myself, so that any suggestion is valuable .... and I'm not touchy .... a question why byteavaible can be "0" (as it seems to be in fact) ... ie what mechanism produces this? .... you can also send me links where it is explained .... in my head the sockets work with datagrams of predefined size and with a minimum of control .... but I understand that it's just my fantasy .. ..

Christian Ehrlicher

A tcp stream is a ... stream.
So you have to define your own protocol to send data over your stream and be able to decrypt it on the receiver side. Or you use e.g. QDataStream which implements a protocol you can use. There are a lot of questions and answers about this in the forum -> forum search

ChrisW67

@gfxx said in send sqlite database file trought tcp:

a question why byteavaible can be "0" (as it seems to be in fact) ... ie what mechanism produces this? ....

Sender sends 20000 bytes. This will go across the network in a series of packets of up to a network path specific maximum size. Let's say 1460 bytes, a typical maximum segment size in TCP.
Receiver receives some of these bytes.
The bytesAvailable() call returns 1460 and you read them before looping back.
If you are lucky then more of the 20000 bytes has arrived in the short time it took to read and save the first chunk, and you can read, save, and loop again.
If you are not lucky then no more of the 20000 bytes has arrived and your bytesAvailable() call returns zero.

gfxx

@ChrisW67 said in send sqlite database file trought tcp:

Sender sends 20000 bytes. This will go across the network in a series of packets of up to a network path specific maximum size. Let's say 1460 bytes, a typical maximum segment size in TCP.
Receiver receives some of these bytes.
The bytesAvailable() call returns 1460 and you read them before looping back.
If you are lucky then more of the 20000 bytes has arrived in the short time it took to read and save the first chunk, and you can read, save, and loop again.
If you are not lucky then no more of the 20000 bytes has arrived and your bytesAvailable() call returns zero.

real thanks ... good explain.

gfxx

@Christian-Ehrlicher

completely missing ....

	if (client->bytesAvailable() <=0)    
		return;

think too tired......