QSignalSpy report wrong number of emitted signal

sardylan

Hi all,
I'm refactoring one of my early Qt5 project, which aim to be an editor for channel memories in a '90 radio devices, used nowadays in Ham radio community.
Channels informations (with frequency, power and other settings) are memorized in a small EEPROM (2kb of data).
My software uses a custom class to parse and serialize data from a QByteArray, using simple getter and setter methods.

Each setter compute the offset to detect the right byte to adjust, compute the new value for the single byte, and replace it using a method that take care of replace the byte and emit a byteUpdated signal:

class EEPROM {

[...]

    private:
        QByteArray data;

        void assign(int pos, quint8 value);

    signals:
        void byteUpdated(int pos, quint8 value);

};

void EEPROM::assign(int pos, quint8 value) {
    data.replace(pos, 1, (const char *) &value, 1);

    QMetaObject::invokeMethod(
            this,
            "byteUpdated",
            Qt::QueuedConnection,
            Q_ARG(int, pos),
            Q_ARG(quint8, value)
    );
}

I'm trying to write some tests to keep under control every single operation. Everything seems to work fine except signals.
Using QSignalSpy I verify that byteUpdated signal is emitted once per every single byte replaced.
Tests run normally in first 4 test methods, starting from which the count of signals is wrong.

Here there is output of test command:

********* Start testing of qfm1000::eeprom::EEPROMTest *********
Config: Using QtTest library 5.14.2, Qt 5.14.2 (x86_64-little_endian-lp64 shared (dynamic) release build; by GCC 10.2.0)
PASS   : qfm1000::eeprom::EEPROMTest::initTestCase()
PASS   : qfm1000::eeprom::EEPROMTest::clear()
PASS   : qfm1000::eeprom::EEPROMTest::frequencyBand()
PASS   : qfm1000::eeprom::EEPROMTest::channelRxFreq()
PASS   : qfm1000::eeprom::EEPROMTest::channelRxFreqBytes()
PASS   : qfm1000::eeprom::EEPROMTest::channelTxFreq()
PASS   : qfm1000::eeprom::EEPROMTest::channelTxFreqBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::channelRxCtcss() Compared values are not the same
   Actual   (signalSpy.size()): 201
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(165)]
PASS   : qfm1000::eeprom::EEPROMTest::channelRxCtcssBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::channelTxCtcss() Compared values are not the same
   Actual   (signalSpy.size()): 101
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(193)]
PASS   : qfm1000::eeprom::EEPROMTest::channelTxCtcssBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::channelPower() Compared values are not the same
   Actual   (signalSpy.size()): 101
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(221)]
PASS   : qfm1000::eeprom::EEPROMTest::channelPowerBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::channelSquelch() Compared values are not the same
   Actual   (signalSpy.size()): 501
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(282)]
PASS   : qfm1000::eeprom::EEPROMTest::channelSquelchBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::channelSelectiveCalling() Compared values are not the same
   Actual   (signalSpy.size()): 601
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(351)]
PASS   : qfm1000::eeprom::EEPROMTest::channelSelectiveCallingBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::channelCpuOffset() Compared values are not the same
   Actual   (signalSpy.size()): 201
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(388)]
PASS   : qfm1000::eeprom::EEPROMTest::channelCpuOffsetBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::startupChannel() Compared values are not the same
   Actual   (signalSpy.size()): 201
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(424)]
PASS   : qfm1000::eeprom::EEPROMTest::startupChannelBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::keyBeep() Compared values are not the same
   Actual   (signalSpy.size()): 2
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(445)]
PASS   : qfm1000::eeprom::EEPROMTest::keyBeepBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::tot() 'signalSpy.wait()' returned FALSE. ()
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(467)]
PASS   : qfm1000::eeprom::EEPROMTest::totBytes()
FAIL!  : qfm1000::eeprom::EEPROMTest::lowPower() Compared values are not the same
   Actual   (signalSpy.size()): 2
   Expected (1)               : 1
   Loc: [/home/sardylan/develop/sardylan/qfm1000/src/eeprom/test/eeprom.cpp(488)]
PASS   : qfm1000::eeprom::EEPROMTest::lowPowerBytes()
PASS   : qfm1000::eeprom::EEPROMTest::cleanupTestCase()
Totals: 18 passed, 10 failed, 0 skipped, 0 blacklisted, 5266ms
********* Finished testing of qfm1000::eeprom::EEPROMTest *********

Process finished with exit code 10

Number are not random, all wrong actual values are always the same every time I run tests, and they begin to be wrong always at the same method.
I tried using both a class-scope and method-scope instance of QSignalSpy, I tried to add clear() and also tried to add registration to metatypes, with no result.
Perhaps I'm using tests in a wrong way, or tests are running with threads and there is some non thread-safe code.
Can you help me finding the problem?

Source code is fully available at https://github.com/sardylan/qfm1000/tree/remake/src/eeprom, branch "remake".

Many thanks

SGaist

Hi,

Is bytesUpdated connected to a logic that triggers again assign ?

sardylan

Hi SGaist,
thanks for answering.
bytesUpdated is not connected.
The idea is to use it to refresh user interface, but now that I'm using it only with tests, the only interaction with the signal is the QSignalSpy instances on code.

sardylan

I managed to solve the problem.
The cause was my mistake in understanding how signals works in test environment.

During tests I have no EventLoop running, so the signal-slot mechanism of Qt can't work.
De facto, every time the assign method execute, I was emitting a signal using a Qt:QueueConnection, but there was no EventLoop running to process the signal queue.

Every time I call a setter method in test code, the next line have to be:

QVERIFY(signalSpy->wait());

which waits, no more than 5 seconds, running an internal EventLoop to process all emitted signals, and waiting for signals of the same type declared in QSignalSpy instance.

Since I had some methods with signal checks and some with only simple code checks (but both of them emit signals) I had a continuous jump between methods with and without QSignalSpy usage.

Since no EventLoop is running on tests, when I called the wait method on QSignalSpy instance, I was getting all the queued signals emitted on the previous test method, which also explain why I was getting constant numbers on signal counts.
I was not able to resolve just calling signalSpy->clear() in cleanup method because there was nothing to clear.

One solution could be to run the EventLoop between every tests method, just to be sure that the next test will handle only signals emitted in its code.
I resolved the problem implementing signals checks in every test method, which not only was my initial goal, but which ensure that every signal-emitting code is followed by a call to wait() method, which runs the EventLoop.
Every signal, now, is kept inside the test method and everything seems to works.

Thanks for support