QTcpServer : Maximum number of sockets ?

k.gaitanis

I'm writing a TCP Server web service which would need to have about 10000 simultaneous open connections. When running my tests it seems that the maximum number of allowed connections is somewhere near 1024.

I have written a minimal server example which simply accepts the incoming connections and a minimal client test app which creates 400 clients and connects to the server (no data exchange). The client test app is launched several times (even from different computers) to see how many clients will manage to connect. The server is always limited in the number of connections without producing any easily catchable errors. This has been tested on Mac and Linux with almost the same results.

On Mac there is an error printed repeatedly on the screen : "select: Invalid argument" and the server stops responding. On Linux there is nothing printed on the screen but the server also stops responding. The exact number of maximum connections depends on which computer I'm running the server (997 on my mac laptop, 1013 on my linux remote server) and is exactly the same every time I run the test.

My question is : Is this limitation coming from Qt or from somewhere else? Have you tested QTcpServer with thousands of connections? If so, where can I find an example of that ?

My test code is really minimal but if you think I'm doing something wrong I can post it.

mcosta

HI,

the problem is inside the EventDispatcher implementation.

On Mac is by default the class QEventDispatcherUNIX that uses select() to monitor activities of sockets and other file descriptor and select() doesn't not support more than 1024 file descriptors (actually doesn't support a fd ID higher than 1024)

On Linux by default the event dispatcher is implemented using glib and IIUC it uses poll(). In that case the limit is higher (about 10240 fd) but with thousands of connections is very slow. (in case glib is not available it uses select())

Possible solutions??

Don't use a single process to handle all the sockets
Write your own event dispatcher (best solution is to use kevent for Mac and epoll for Linux)
Don't use Qt for that

NOTE: We had the same problem and the solution was:

Use an external process to handle all connections without using Qt (we used ZeroMQ)
Share information with the Qt process using a single connection