Read file untill EOT in client server programming
-
@hjohn
What exactly does your "EOT" mean? (If by any chance you meant "EOF" this is different.)If you are sending multiple files and want to receive them such that you know where each one ends, you must put some "marker" of some kind between each file so that the server knows where each one ends.
Since you are reading text files you could put in some character which you know "cannot" occur (e.g. maybe Ctrl+Z). However, the usual way of transferring multiple files is to precede each file with some "header" information, such as the number of bytes to expect till end-of-file, and build your protocol based on that.
If you do it with whatever your "EOT" marker is, however, you cannot go
socket.readAll()and then expect justendsWith(EOT)to pick out that marker. You must allow for the possibility that the end of one file and the beginning of the next file are received in a single "packet" at the server side, so the "EOT" is in the middle of a "chunk", not necessarily right at the end. You have to code correctly for that.... -
@JonB "EOT is 0x04.
"Can i use any alternative of readALL() or after receiving all data using readall().I separate them by checking whether it contains EOT or not.if so then up to that marker store data into one file and another data into second file.@hjohn
OK, so you're using the UNIX^Dto terminate your files.And you're actually explicitly sending that character from client to server at the end of each file, right?
So all you can rely is that somewhere in the blocks returned from
readAll()there will be an0x04every so often, but not necessarily right at the end of a "chunk" received. You must search the whole of the text for that character, not just the end of it. Furthermore, it might be that that a single read actually returns multiple files each separated by their0x04s in one go, so you must loop to (potentially) pick out each separator. -
Not perfect (only works if all the data is ASCII chars and it's hugely inefficient) but gives you an idea
socket.startTransaction(); QByteArray alldata = socket.readAll(); socket.rollbackTransaction(); int lastValidEOT=-1; for(int eotIndex = alldata.indexOf((char)4);eotIndex >= 0;eotIndex = alldata.indexOf((char)4)){ lastValidEOT=eotIndex B = alldata.left(eotIndex); alldata.remove(0,eotIndex+1); qDebug() << B; } if(lastValidEOT>=0) socket.skip(lastValidEOT+1);Edit: efficiency improved
-
Not perfect (only works if all the data is ASCII chars and it's hugely inefficient) but gives you an idea
socket.startTransaction(); QByteArray alldata = socket.readAll(); socket.rollbackTransaction(); int lastValidEOT=-1; for(int eotIndex = alldata.indexOf((char)4);eotIndex >= 0;eotIndex = alldata.indexOf((char)4)){ lastValidEOT=eotIndex B = alldata.left(eotIndex); alldata.remove(0,eotIndex+1); qDebug() << B; } if(lastValidEOT>=0) socket.skip(lastValidEOT+1);Edit: efficiency improved
-
@hjohn
OK, so you're using the UNIX^Dto terminate your files.And you're actually explicitly sending that character from client to server at the end of each file, right?
So all you can rely is that somewhere in the blocks returned from
readAll()there will be an0x04every so often, but not necessarily right at the end of a "chunk" received. You must search the whole of the text for that character, not just the end of it. Furthermore, it might be that that a single read actually returns multiple files each separated by their0x04s in one go, so you must loop to (potentially) pick out each separator. -
In the another thing i want your help is,I am sending file's data continuously to server from client.when STOP button clicked,It should stop sending file's data.
I am sending initially 1000 files data.Before I press STOP button the process of sending files's data has already completed.so,I want to send data after some amount of time so I get a chance to press STOP button.what should i applied? -
@JonB yeah i have applied same way as u have suggested.
But Instead of getting all files into buffer chunk, I want to receive it one by one into buffer.@hjohn said in Read file untill EOT in client server programming:
But Instead of getting all files into buffer chunk, I want to receive it one by one into buffer.
Then you'd better write to Mr TCP/IP/socket and tell him you don't like the way it works and would like him to change the design :)
TCP/IP/sockets use a continuous-streaming protocol, without record boundaries, so you can't have it your way, I'm afraid. You have two choices:
-
Current way. It really isn't too difficult to recognise these
^Dboundaries and split into separate files on that basis. You even have @VRonin's clever but lazy way of doing it to reduce the code you need. -
You change your protocol between client & server such that after sending one file (terminated by the
^D) the client sends an "acknowledgement" message back to the server; the server waits for that before starting to send the next file. This way the client sees its buffer only ever filled with one file, it never sees a second file till it has finished with the previous file. Now you can use your originalwhile(!B.endswith(EOT)). This approach is a touch slower, if you were sending a large number of small files.
-
-
@hjohn said in Read file untill EOT in client server programming:
But Instead of getting all files into buffer chunk, I want to receive it one by one into buffer.
Then you'd better write to Mr TCP/IP/socket and tell him you don't like the way it works and would like him to change the design :)
TCP/IP/sockets use a continuous-streaming protocol, without record boundaries, so you can't have it your way, I'm afraid. You have two choices:
-
Current way. It really isn't too difficult to recognise these
^Dboundaries and split into separate files on that basis. You even have @VRonin's clever but lazy way of doing it to reduce the code you need. -
You change your protocol between client & server such that after sending one file (terminated by the
^D) the client sends an "acknowledgement" message back to the server; the server waits for that before starting to send the next file. This way the client sees its buffer only ever filled with one file, it never sees a second file till it has finished with the previous file. Now you can use your originalwhile(!B.endswith(EOT)). This approach is a touch slower, if you were sending a large number of small files.
@JonB yeah U r right.
@hjohn said in Read file untill EOT in client server programming:
In the another thing i want your help is,I am sending file's data continuously to server from client.when STOP button clicked,It should stop sending file's data.
I am sending initially 1000 files data.Before I press STOP button the process of sending files's data has already completed.so,I want to send data after some amount of time so I get a chance to press STOP button.what should i applied?Can u help me on this!!
-
-
In the another thing i want your help is,I am sending file's data continuously to server from client.when STOP button clicked,It should stop sending file's data.
I am sending initially 1000 files data.Before I press STOP button the process of sending files's data has already completed.so,I want to send data after some amount of time so I get a chance to press STOP button.what should i applied?@hjohn said in Read file untill EOT in client server programming:
I want to send data after some amount of time so I get a chance to press STOP button.what should i applied?
Now this does get tricky, precisely because of the continuous streaming protocol. There isn't really a "natural" way to do this, you have write code to implement whatever you want. I see 3 possibilities:
-
I've never had to do it myself, but I believe the way you can do this is that sockets support something called an "out-of-band" ("OOB") message, which your client can send to your server. I do not know whether Qt's functions support this (haven't looked). This is a message which gets through on a separate channel ahead of any data already sent. Your server sees this and (you write code so that it) abandons receiving from the client.
-
If you can afford to, you only check for "STOP" at the server between each complete file send. If pressed, you don't send the next file, so there are no "incomplete file sends". Of course, this means that "STOP" is not acted upon immediately. If your files are small this may be OK; but if you were in the middle of sending a huge file there would be a considerable delay while that one finished sending before the transfer gets terminated.
-
You are already using a special character of
^Dto mark the boundary between files. You assume that character can never occur in the data. You could extend this to pick another special character which the server sends on "STOP", say^Zor\000or something. Now your server, which is already scanning for the^Ds, also checks whether the latest character is that special "STOP" marker, and abandons if it sees it. In principle this protocol is "instantaneous", there is no delay.
-
-
@hjohn said in Read file untill EOT in client server programming:
I want to send data after some amount of time so I get a chance to press STOP button.what should i applied?
Now this does get tricky, precisely because of the continuous streaming protocol. There isn't really a "natural" way to do this, you have write code to implement whatever you want. I see 3 possibilities:
-
I've never had to do it myself, but I believe the way you can do this is that sockets support something called an "out-of-band" ("OOB") message, which your client can send to your server. I do not know whether Qt's functions support this (haven't looked). This is a message which gets through on a separate channel ahead of any data already sent. Your server sees this and (you write code so that it) abandons receiving from the client.
-
If you can afford to, you only check for "STOP" at the server between each complete file send. If pressed, you don't send the next file, so there are no "incomplete file sends". Of course, this means that "STOP" is not acted upon immediately. If your files are small this may be OK; but if you were in the middle of sending a huge file there would be a considerable delay while that one finished sending before the transfer gets terminated.
-
You are already using a special character of
^Dto mark the boundary between files. You assume that character can never occur in the data. You could extend this to pick another special character which the server sends on "STOP", say^Zor\000or something. Now your server, which is already scanning for the^Ds, also checks whether the latest character is that special "STOP" marker, and abandons if it sees it. In principle this protocol is "instantaneous", there is no delay.
-
-
@JonB I have implemented for large amount of files(1000).In this case,when I clicked STOP button of client GUI,I works only after sending 1000 files.
Not working immediately.@hjohn
I have offered you 3 approaches above which will work. You need to implement one of them.Just to be clear: because you need to see the "STOP" when it's pressed, depending on your implementation you may need the client's file transfer to be taking place in its own thread, so that the main GUI thread (where "STOP" gets recognised) does not get blocked by the socket transfer.
