copy entire SQLite DB trought soket

[Christian Ehrlicher]

@gfxx said in copy entire SQLite DB trought soket:

why is not needed thread

The same reason which is stated in every topic here when someone without basic knowledge about threads is using threads in Qt with e.g. QProcess, Q(Tcp/Ud/whatever)Socket, Qfoo- Qt system is asynchronous signals and slots is the way to go to not mess around with threading issues for no reasons. There is no QThread usage in QProcess/QTcpSocket/... documentation so why use threads? Because it sounds fancy I would guess...

[gfxx]

@Christian-Ehrlicher said in copy entire SQLite DB trought soket:

The same reason which is stated in every topic here when someone without basic knowledge about threads is using threads in Qt with e.g. QProcess, Q(Tcp/Ud/whatever)Socket, Qfoo- Qt system is asynchronous signals and slots is the way to go to not mess around with threading issues for no reasons. There is no QThread usage in QProcess/QTcpSocket/... documentation so why use threads? Because it sounds fancy I would guess...

Not for sure ..... normally I use qthread for OpenCv calculation only or for modbus/canbus operation, so in my non-comproved ideas when use

waitForReadyRead(5000)

I immagine is better use qthread instead mainwindows Gui .... but I think you learn me that these waiting is not blocking on mainwindows GUI so I can perform it on mainwindows .... Is right these?

thanks for these advice.
bkt

[JonB]

@gfxx
The whole point is not to use the waitFor...() methods, use only the signals & slots for sockets. Then your UI does not block and freeze, and you do not need a separate thread, At least on the client side.

[gfxx]

@JonB thanks .... but on CLIENT side there are not Qthread .... I use signal&slot only on main tharead aka mainwindows ..... on SERVER side (other GUI app with mainwindows too) I use Qthread because in my original ideas we connect more than one client, and every 50-60 min need to copy for every client a little DB from client TO server ... so sever can have an identical copy of single DB every hour .... I means 10 client max .... server app must manage the data trought QChart .... and data need to be processed .... similar data for 10 client print on same QChartView .... i have many QChartView .... so think is better to have local data instead remote query ....
So if in your experience and knowelge, is preferred to use SIGNAL&SLOT at SERVER side mainwindows GUI .... I work on it for sure .... because less code for make same things .... less is better .... appreciate you opinion on these sketch of project .... : is better manage signal & slot from mainwindows in these situation too? No freeze gui. Perfect I love it. But is possible? Sorry but never work with soket so really not have experience on these .... and I'm intrested on your opinion. If possible i change immediately strategy and abandone Qthread.

regards
bkt

[JonB]

@gfxx
I mentioned about for server because you can look at examples Fortune Server Example versus Threaded Fortune Server Example.

Your question is way too complicated/vague for people to answer specifically. But if @SGaist & @Christian-Ehrlicher are saying they do not think you need threading on the server side that is good for me. One thing you should not do is use threads and change to using blocking calls in them, that is just a waste, when Qt's signal/slots are non-blocking and do not need separate threads.

[gfxx]

@JonB said in copy entire SQLite DB trought soket:

Your question is way too complicated/vague for people to answer specifically. But if @SGaist & @Christian-Ehrlicher are saying they do not think you need threading on the server side that is good for me. One thing you should not do is use threads and change to using blocking calls in them, that is just a waste, when Qt's signal/slots are non-blocking and do not need separate threads.

thanks ..... understand the point .... for certain I have a one more doubt: can connect multiple client on one QTcpserver & socket at same time? or need to start more than one server (1 server to one client? or can be more client to one server + socket?) .... these because I read a lot of post these week and some they confused me a bit.....

[Christian Ehrlicher]

@gfxx said in copy entire SQLite DB trought soket:

can connect multiple client on one QTcpserver & socket at same time?

Yes, that's the task for a server.
QTcpServer::newConnection()

[SGaist]

Out of curiosity, why do you want to copy database files from multiple clients to a server ?

It sounds like you are trying reinvent networked database system like PostgreSQL.

[gfxx]

@SGaist said in copy entire SQLite DB trought soket:

Out of curiosity, why do you want to copy database files from multiple clients to a server ?
It sounds like you are trying reinvent networked database system like PostgreSQL.

Because on machinery there are installed SQLite db ..... and it is not networked database .... actually I evaluate if is more convenient change totally database system on machinery or try to copy database on machinery (as try in these moment) ..... plus server is not everytime power ON so indipendent database on single machine prevent loss of data and can work in disconnected mode without issue.

[JonB]

@gfxx said in copy entire SQLite DB trought soket:

and every 50-60 min need to copy for every client a little DB from client TO server ... so sever can have an identical copy of single DB every hour

Assuming users are connected to their SQLite database files and you copy these files while they may be updated or in partial state I'm not sure you will get "reliable" backups?

[SGaist]

Then you should rather implement a proper backup strategy. Having a bunch of machines with each their own SQLite file that you copy at some random time is not going to provide the resiliency you seem to be after.

You do not need to have one central server for all your machines to connect to if it's not possible but you really need to study the architecture of your setup.

[gfxx]

@JonB @SGaist ..... because the real control is on client .... the simply base ideas was client that decide when send copy of DB file .... example when no one read or write on client sqlite DB. These is quite easy .... because soft plc is connected on qt app that runs on realtime system. Plc write on DB, so plc can decide when send DB copy without c++ strategy for copy ... plc decide if qt app can write or read on DB .... and can block writing and readig action ..... more safe more easy and vice versa ;) .....

[gfxx]

Any how now my Qtcpserver work on mainwindos .... now I have some issue on managin send and receive messages (a sort off proto-protocol for exchange data) ..... based on other post I subclass QTcpSocket for have trace of every client that have connection... is the right strategy or better is to use "nextpendingconnection" macro only? what you can suggest based on your knowelge?

[SGaist]

It's not a macro it's a function. Anyway, if you have a protocol that requires multiple communication between your client and server you will have to manage a list of connections.

You should also properly model the communication and maybe use a state machine to handle the various steps required from start to finish of your database copy.

[gfxx]

@SGaist said in copy entire SQLite DB trought soket:

It's not a macro it's a function. Anyway, if you have a protocol that requires multiple communication between your client and server you will have to manage a list of connections.

Ok noted ... thanks

You should also properly model the communication and maybe use a state machine to handle the various steps required from start to finish of your database copy.

Not really understand .... never use state machines but know it .... but why need to use state machine? Why need various step? Explain me better please .... in my ideas, one time client choose that is ok to copy DBfile, send a messages to server for advice about data transfer .... server send "ok go" messages and client send.

So QByteArray sender = file.readAll() client side ..... and server side ...

    while (client->bytesAvailable()){
        QByteArray buffer;
        buffer.resize( client->bytesAvailable() );
        client->read( buffer.data(), buffer.size() );
        if(buffer.size() < 150){
            qDebug() << "Server is receiving ...."<<buffer;
        }
        else{
            QFile filew("/home/mypc/.DBmem/DB_mine.sqlite3");
            if(!(filew.open(QIODevice::Append)))
            {
                qDebug("....file not open....");
            }
            else{
                filew.write(buffer);
                //filew.
                filew.close();
                qDebug("... write the file ....");
            }

        }
    }

perhaps .... now work ok and file is received in right dimension (667,7Kb like client side) ..... but sqlite browser send these messages: FILE COULD NOT BE OPEN: FILE IS NOT A DATABASE ..... but not know why .... strange because is exactly same dimension than client side ....

client side file is write on socket in these way:

QFile file("/home/mypc/.DBmem/DB_mine.sqlite3");
if (!file.open(QFile::ReadOnly)){
        qDebug() << "File not open .....";
    }
 else
}
    QByteArray mydata = file.readAll();
    QByteArray block;
    QDataStream stOut(&block, QIODevice::WriteOnly);
    stOut.setVersion(QDataStream::Qt_5_15);
    stOut << (quint32)0;
    stOut << mydata;
    stOut.device()->seek(0);
    stOut << quint32((block.size() -sizeof(quint32)));;
    tcpSocket->write(block);
}
file.close();

file client side is 667,7kb ... and server side the same ... but server side could not open ..... I'm in error about file reading client side? ... need to use QtextStream?

Appreciate any help.

regards
bkt

[SGaist]

You are not handling the block size you are sending server side.

As for the state machine, it allows you to automate the protocole handling. It's not mandatory though.

That said, you don't seem to do any error handling.

[gfxx]

@SGaist said in copy entire SQLite DB trought soket:

ou are not handling the block size you are sending server side.

Ok thanks ... you means in client code? or on server code?... because client send and server receive .....and my english is not good enought :))

As for the state machine, it allows you to automate the protocole handling. It's not mandatory though.

oh server side ... like plc do client side .... but can start a soft plc server side too for that ... i Think is the same result .... you means that right?

That said, you don't seem to do any error handling.

You are in right ... because it is only self learning code .... but think is better add that because maybe i loose something .....

[SGaist]

Server side. But as I already suggested: use a transaction to transfert your data.

[gfxx]

@SGaist you means somethings like the pieces of code?

void Reader::slotOnReadyRead()
{
    mStream.startTransaction();

    quint64 availableBytesForReading = mStream.device()->bytesAvailable();
    QByteArray binaryDataBlock;
    char *tmp = new char[availableBytesForReading];
    mStream.readRawData(tmp, availableBytesForReading);
    binaryDataBlock.append(tmp, availableBytesForReading);
    delete[] tmp;
    tmp = nullptr;

    if (mStream.commitTransaction())
    {
        emit signalNewData(binaryDataBlock);
    }
}

not my code only an example and not try it.

[SGaist]

The fortune client example implements that in a simpler manner.