QSslSocket::readyRead() never triggered with Python server

[ArtiFAS]

Hi all!

I'm developping a desktop app with Qt6.4.2 in C++, which is a client to a python "server" (hosted on a RockPi E). I'm currently trying to implement a TLSv1.3 communication between both of them with custom messages (headers and payload all based on Json over TCP).

An EthernetComm class is instanciated within the app's run() method, which contains an EthernetReceiver() class and an EthernetSender() class, which both receives the QSslSocket* created in the EthernetComm class as argument of their constructor.

As of now, the server receives all of the "commands" sent by the client (Qt app on CLI for now). The commands are sent from a custom-made producer-consummer threaded structure (EthernetSender), which I've tested toroughly within many projects. The sending structure works flawlessly (the python server receives all sent data without any problem).

The real issue comes from the EthernetReceiver class. A slot is connected to the QSslSocket::readyRead() signal in the EthernetReceiver, but is never triggered when receiving data from the server. The EthernetReceiver also uses a Producer-Consummer structure, as the EthernetSender does.

Here are the relevant classes to my issue (without the .h files, which would be redundant):

EthernetComm.cpp

void EthernetComm::sendCommand(Command command)
{
	_sender->sendCommand(command);
}

void EthernetComm::socketError(QAbstractSocket::SocketError)
{
	qDebug() << "Received a socket error: " << _socket->errorString();
}

void EthernetComm::sslErrors(const QList<QSslError> &errors)
{
	qDebug() << "Received " << errors.size() << " SSL error(s): ";
	for (const auto &error : errors)
	{
		qDebug() << "	" << errors.front().errorString();
	}
}

void EthernetComm::socketEncrypted()
{
	qDebug() << "SSL socket is now encrypted with " << _socket->sessionCipher();
}

void EthernetComm::socketStateChanged(QAbstractSocket::SocketState state)
{
	qDebug() << "SSL socket state changed to " << state;
}

EthernetComm::EthernetComm()
{
	qDebug() << "SSL Library Build Version: " << QSslSocket::sslLibraryBuildVersionString();
	qDebug() << "SSL Supported: " << QSslSocket::supportsSsl();
	qDebug() << "SSL Library Version: " << QSslSocket::sslLibraryVersionString();

	QString path = qgetenv("LSPDD_FILES");

	if (!_socket) _socket = new QSslSocket();

	QSslConfiguration config;
	QFile pkcs(path + "/cert/ca.pfx");

	pkcs.open(QFile::ReadOnly);
	QSslKey key;
	QSslCertificate caCert;

	static bool import = QSslCertificate::importPkcs12(&pkcs, &key, &caCert);

	pkcs.close();

	if (!import)
	{
		qDebug() << "(FATAL ERROR) Failed to load PKCS12 infos, exiting";
		exit(-1);
	}

	//config.setPrivateKey(key);
	config.addCaCertificate(caCert);
	config.setPeerVerifyMode(QSslSocket::VerifyNone);
    config.setProtocol(QSsl::TlsV1_2OrLater);
	_socket->setSslConfiguration(config);

	_receiver = new EthernetReceiver(_socket);
	_sender = new EthernetSender(_socket);

	connect(_socket, &QSslSocket::errorOccurred,
			this, &EthernetComm::socketError);

	connect(_socket, QOverload<const QList<QSslError> &>::of(&QSslSocket::sslErrors),
			this, &EthernetComm::sslErrors);

	connect(_socket, &QSslSocket::stateChanged,
			this, &EthernetComm::socketStateChanged);

	connect(_socket, &QSslSocket::encrypted,
			this, &EthernetComm::socketEncrypted);

	_socket->connectToHostEncrypted(_hostname, _port);

    if (!_socket->waitForEncrypted(3000)) {
        _socket->close();
        qDebug() << "Closing socket, not encrypted";
        exit(-1);
    }
}

EthernetSender.cpp

void EthernetSender::sendCommand(const Command& command)
{
	std::lock_guard<std::mutex> lock(_mutex);
	qDebug() << "\nEthernetSender::sendCommand : Pushed a task to the queue";
	_taskQueue.push(command);
	_dataCond.notify_one();
	_timer->start();
}

void EthernetSender::processCommands() {
	while (_shouldRun)
	{
		std::unique_lock<std::mutex> lock(_mutex);
		_dataCond.wait(
			lock, [this]{return !_shouldRun || !_taskQueue.empty();}
		);

		if (!_shouldRun) break;

		Command command = _taskQueue.front();
		_taskQueue.pop();
		lock.unlock();

		ethernetSend(command);
	}
}

EthernetSender::EthernetSender(QSslSocket* socket)
{
	_socket = socket;

	_shouldRun = true;

	_threadQueue.emplace_back(&EthernetSender::processCommands, this);
}

EthernetReceiver.cpp

void EthernetReceiver::processData() {
	while (_shouldRun) {
		std::unique_lock<std::mutex> lock(_mutex);
		_dataCond.wait(
				lock, [this] { return !_shouldRun || !_dataQueue.empty(); }
		);

		if (!_shouldRun) break;

		QByteArray data = _dataQueue.front();
		_dataQueue.pop();
		lock.unlock();

		parseMessage(data); // Method parsing the received data
	}
}


void EthernetReceiver::socketReadyRead()
{
	QByteArray data = _socket->readAll(); //May need to change read logic
	_socket->flush();
	qDebug() << "EthernetReceiver::socketSendReady : Received a message on the socket: " << data;

	std::lock_guard<std::mutex> lock(_mutex);
	_dataQueue.push(data);
	_dataCond.notify_one();
	qDebug() << "EthernetReceiver::socketSendReady : Pushed a task to the queue";
}

EthernetReceiver::EthernetReceiver(QSslSocket* socket)
{
	_shouldRun = true;
	_threadQueue.emplace_back(&EthernetReceiver::processData, this);

	_socket = socket;

	connect(_socket, &QSslSocket::readyRead,
			this, &EthernetReceiver::socketReadyRead);
}

Nothing is ever printed from the EthernetReceiver class... The readReady signal is never triggered.

It may be useful to precise that the server is on the same subnet as the client and both can ping each other.

A python script as also been made to emulate the client and it receives all of the server's messages (50 in total), while the Qt client receives 2 or 3, after what the server stops to send any data (due to the TCP connection I would assume). I have a funny feeling that the problem is caused by a network issue, not a code problem. I'm still trying to have all of your inputs since I'm kinda desperate. Thanks in advance, hope some of y'all might have a clue of how to solve my issue!

[SGaist]

Hi and welcome to devnet,

Did you check with a tool like Wireshark to see what happens on the network ?

[ArtiFAS]

@SGaist Yes I did,

I see all of the TLS Handshake being done successfully.

The server then sends data to the client after a client request and it looks like the client acknowledges those packets.

The IP .19 is the client and the .103 is the server. There are many SSH packets since my client has an ssh session opened with the server. The ports are the right ones.

There's a fourth screenshot of the packets,but I'm not sure it is relevant.

[Kent-Dorfman]

@ArtiFAS

Just being nitpicky, but technically it's not an ethernet receiver. There is no requirement that it have anything to do with ethernet. The SSL connection is over TCP (layer 4) and ethernet is a layer (1/2) service. IP need not be carried over ethernet.

I think you may be making it too complicated by using threads...and I question the need for mutex locks if you get rid of the threads.

If you need to service multiple connections, the long standing UNIX way is to fork a new instance of your program for each connection.

[jsulm]

@Kent-Dorfman said in QSslSocket::readyRead() never triggered with Python server:

the long standing UNIX way is to fork a new instance of your program for each connection

The Qt way is to use assynchronous nature of Qt with signals/slots :-)

[Kent-Dorfman]

@jsulm said in QSslSocket::readyRead() never triggered with Python server:

The Qt way is to use assynchronous nature of Qt with signals/slots :-)

SPOILER!

Haven't you heard of making the student to it by brute force before showing them the easy way?

[SGaist]

@Kent-Dorfman said in QSslSocket::readyRead() never triggered with Python server:

@jsulm said in QSslSocket::readyRead() never triggered with Python server:

The Qt way is to use assynchronous nature of Qt with signals/slots :-)

SPOILER!

Haven't you heard of making the student to it by brute force before showing them the easy way?

Well, I am not a fan of teaching the youngster about the screwdriver after they broke their fingers with a sledgehammer.

[Kent-Dorfman]

@SGaist said in QSslSocket::readyRead() never triggered with Python server:

Well, I am not a fan of teaching the youngster about the screwdriver after they broke their fingers with a sledgehammer.

Eh, I wouldn't consider that to be a fair analogy. fork() is easier to implement, and teaches more fundemental theory, while being less efficient.

[SGaist]

@Kent-Dorfman agreed, but when you use a framework, learn the framework and what it provides rather than try to make it fit your habits.
Since it's Qt, the networking stuff rarely needs any threading or forking.

[ArtiFAS]

@Kent-Dorfman
Agreed, it could be used with any Data-Link Layer protocol, but in this specific project, it is only used over Ethernet, hence the name choice.

I've decided to thread the Sender as a personal challenge, but I understand it's more complicated than it could be, I'm totally fine with it.

As for the Receiver, all messages are received asynchronously (at any given moment, the Rock Pi E, the server, could send a new message on the socket). That is why I decided to thread this bit. The Rock Pi can also send big payloads (100MB) separated in many smaller messages (as it should). Parsing them is too much of load for the signal's thread, so I implemented my own Producer-Consumer structure, which I think is fair enough in this case.

As for the forks, I don't think they're totally appropriate for what I'm trying to achieve. The app's main goal is efficiency and lightness. I will explore this avenue, but I doubt it's the easiest/fastest way to achieve my goals since the app must be usable on Windows and Linux.

Thanks for the advice!

[ArtiFAS]

I've dig up a little more and found that using the QSslSocket::waitForReadyRead() method 5 secs after sending the message actually returns true. I then read the socket with socket->readAll() and messages have been received (they're valid!). The signal still isn't being triggered though... Must waitForReadyRead be called for the signal to be triggered? I've read that these are separate, but I'm not sure about their behavior...

[SGaist]

One of the issue is that you have while loop that will block Qt's event loop processing. Don't use such a loop as Qt is an event driven framework. You should implement proper signal and slot handling so that you trigger the next command sending after the last one. You can use a 0 based QTimer for example.

[ArtiFAS]

@SGaist I'm not totally sure which while loop you are refering to. There's one in the EthernetReceiver::processData() method and one in the EthernetSender::processCommand() method. Both of these methods are called in a separate thread which is woke previously in the EthernetSender::sendCommand(Command) and the EthernetReceiver::socketReadyRead() methods. In both case, the loops are not affecting the return time of the readyRead() signal.

I'm not sure I've understood fully what you're proposing, but let me know if this comment is relevant or not.

[SGaist]

@ArtiFAS That's one of the issue with the code sample you gave. There's not enough context to fully understand how it is working.

Now the question is: why do you need a separate thread to handle your network connection ? Most of the time there's no need for that as Qt is asynchronous.