Async TCP server design

[twinsen]

Hi all,
I'm new to Qt and trying to create a TCP server. I've read the doc for all relevant classes, checked examples and tutorials, but still not quite confident how to implement this. Can you give me a hint how to do this the Qt way?
Here's what my server should do:

accept arbitrary number of TCP clients and pass on messages between them

at connection each client sends a list of messages it's interested in (subscribe)

when a message is received from a client pass it on to everybody who's subscribed for it

clients with slow TCP connection putting/getting large messages should not block/delay others

clients that disappear because of network failure (without proper TCP shutdown) should not block/delay others

So it's more or less broadcast server like thing with the publish/subscribe pattern. I tried doing it with a thread pool (with 4 threads, not 1 thread per client). But QTcpSocket is not a multithreaded thing, and fiddling with the socket form different threads doesn't seem to be a good idea.

What is the recommended way of doing this? Rough guidelines will do. :)
If I implement it with a single network thread (main thread, standard signals/slots), and maybe move subscription lookups (which can be expensive) to a separate thread, will that satisfy requirements 4 and 5?

Any hints appreciated, :)
thanks!

[romankr]

i will share a piece of my experience. But i've been working with winsock - so i don't how to do it with Qt Classes.

The main idea:
The main thread (where you are accepting connections) should be in thread separated from client threads. Each client has it's own thread. Client threads are in one thread pool. Threads interacting with each other by signal/slot mechanism.

every time you accepting connection you should create a separate thread for each client. This approach will solve problems of:

1. accept arbitrary number of TCP clients and pass on messages between them
2. at connection each client sends a list of messages it’s interested in (subscribe)
3. clients with slow TCP connection putting/getting large messages should not block/delay others
4. clients that disappear because of network failure (without proper TCP shutdown) should not block/delay others

one client - one socket - one thread. Interaction with client never goes out of thread. And clients cannot interrupt each other.

The second step - interaction between threads. QT signals/slots is a preferable solution. it will help with

1. accept arbitrary number of TCP clients and pass on messages between them

And QThreadPool will help
3) when a message is received from a client pass it on to everybody who’s subscribed for it

[twinsen]

Hi,
Thanks for the tip. Having one thread per client would be an overkill in this case. The server has about 30 permanent clients. These are idle most of time, and occasional external events trigger bursts of messages between them. This is why I was thinking having only 4 threads to handle the bursts.
I think I'll just implement the most naive single threaded way, and check the performance. And then decide how to continue.

[koahnig]

What you could do is having a container with TcpSockets. Each incoming connection will have its own socket. When you have to broadcast something you go through the container and write the piece of information to every socket. You can check the return value, if the socket is still active. If not, you can remove it.
I am using it for a couple of client connections and it works there. I am not sure if this is the path to go for 30 or more clients.

[twinsen]

Yes, this is the same single threaded way I was thinking of: list of Clients (each Client has a QTcpSocket, and the subscription list). When a message arrives iterate over the list, check the subscription and send the message to the client if appropriate. This is simple and easy, but I'm wondering about the performance.

[koahnig]

What other chances do you have then?
I am using pointers to sockets stored in the containers.

• 3. you might implement different containers for different information/subscription types. You probably may have the socket pointers in different containers. However, the different containers would make only sense when your subscription checking very expensive.
• 4. this should be handled by the socket. I would assume that there is some buffering going on.
• 5. is no problem. As stated I am removing the socket from my container when the transmission has a problem.