Camera acquisition with QStateMachine

go9kata

Dear all,

I am trying to develop a camera capture class with OpenCV and Qt.
A QStateMachine is designed to control the blocking operation of the frame grabbing. Here how it is configures:

DriverVideoCapture::DriverVideoCapture(QObject *parent, const int camid) : QObject(parent)
{
    // configure state machine
    m_engine = new QStateMachine(this);
    QState *state_idle = new QState(m_engine);
    QState *state_acquire = new QState(m_engine);
    QState *state_error = new QState(m_engine);

    state_idle->addTransition(this, &DriverVideoCapture::start, state_acquire);
    state_idle->addTransition(this, &DriverVideoCapture::error, state_error);
    state_acquire->addTransition(this, &DriverVideoCapture::grab, state_acquire);
    state_acquire->addTransition(this, &DriverVideoCapture::stop, state_idle);
    state_acquire->addTransition(this, &DriverVideoCapture::error, state_error);
    connect(state_acquire, &QState::entered, this, &DriverVideoCapture::on_stateAcquire);
    m_engine->setInitialState(state_idle);
    m_engine->start();

    m_videoCapture = new cv::VideoCapture(camid);
    if (!m_videoCapture->isOpened())
        emit error();
}

Inside the state_acquire callback happens the actual frame grabbing.

void DriverVideoCapture::on_stateAcquire()
{
    cv::Mat frame;
    if (!m_videoCapture->isOpened())
        return;

    if (!m_videoCapture->read(frame))
        emit error();

    emit grab();
}

The start and stop signals are connected to UI buttons in the MainWindow class.

m_driverVideoCapture = new DriverVideoCapture(this);
    connect(ui->pushButton_start, &QPushButton::clicked, m_driverVideoCapture, &DriverVideoCapture::start);
    connect(ui->pushButton_stop, &QPushButton::clicked, m_driverVideoCapture, &DriverVideoCapture::stop);

The state machine starts after the start() signal and transit correctly from state_idle to state_acquire. Once in state_acquire it repeatedly transitions to the same state and grabs a frame. The problem comes when the stop() signal is fired. It does not get detected by the StateMachine and it never manages to transition from state_acquire to state_idle again. Any tips, suggestion or remarks on potential pitfalls are very welcomed with that design.
Another question is, if I move the DriverVideoCapture class to a separate thread as a worker thread (as described in QThread manual and moveToThread() method), does the StateMachine get a separate event loop from the main GUI event loop, and will it be responsible only for state machine signals and events (or what ever happens in that class only)?

Thanks you in advance for any responses!

Best,
go9kata

go9kata

Here is my final solution. The key was to call QCoreApplication::processEvents() inside the loop state. This step will enforce processing other signals before re-running the state again.

State machine configuration:

// configure state machine
m_engine = new QStateMachine(this);
QState *m_stateIdle = new QState(m_engine);
QState *m_stateAcquire = new QState(m_engine);
QState *m_stateError = new QState(m_engine);

m_stateIdle->addTransition(this, &DriverVideoCapture::start, m_stateAcquire);
m_stateIdle->addTransition(this, &DriverVideoCapture::error, m_stateError);

m_stateAcquire->addTransition(this, &DriverVideoCapture::stop, m_stateIdle);
m_stateAcquire->addTransition(this, &DriverVideoCapture::frameReady, m_stateAcquire);
m_stateAcquire->addTransition(this, &DriverVideoCapture::error, m_stateError);

connect(m_stateAcquire, &QState::entered, this, &DriverVideoCapture::on_readFrame);

m_engine->setInitialState(m_stateIdle);
m_engine->start();

Callback of looping state_Acquire:

void DriverVideoCapture::on_readFrame()
{
    cv::Mat frame;
    if (!m_videoCapture->read(frame))
        throwError("failed to read frame");

    QCoreApplication::processEvents();

    emit frameReady(frame);
}

SGaist

Hi,

QStateMachine starts its own event loop as explained here.

Shouldn't you rather have a bit more states e.g.:

Acquiring
Acquisition done

That would make it clear what is happening.

go9kata

Thank you for the respond. I moved the class to separate thread and based on description it should get its own event loop.
I tried adding more states too, but it does not really solve the issue.
I still do not fully grasp how QStateMachine deals with signals. From description there are two ways to transition in the machine:

QSignalTransition
QEventTransition

Are QSignalTransitions being scheduled in an event queue too?
I tried to detect the stop() signal with QSignalSpy, but once the state machine is looping in a single state (or it does a targetless transition to state_acquire), the signal is never caught by QSignalSpy, suggesting that the state machine event loop will never try to transition to state_idle (as the behaviour I observe).
I will try to change from QSignalTransition to QEventTransition and introduce a guard in the loop transition.
Any further remarks or clarifications are very welcomed.

SGaist

What are you doing with QSignalSpy ?

go9kata

Here is my final solution. The key was to call QCoreApplication::processEvents() inside the loop state. This step will enforce processing other signals before re-running the state again.

State machine configuration:

// configure state machine
m_engine = new QStateMachine(this);
QState *m_stateIdle = new QState(m_engine);
QState *m_stateAcquire = new QState(m_engine);
QState *m_stateError = new QState(m_engine);

m_stateIdle->addTransition(this, &DriverVideoCapture::start, m_stateAcquire);
m_stateIdle->addTransition(this, &DriverVideoCapture::error, m_stateError);

m_stateAcquire->addTransition(this, &DriverVideoCapture::stop, m_stateIdle);
m_stateAcquire->addTransition(this, &DriverVideoCapture::frameReady, m_stateAcquire);
m_stateAcquire->addTransition(this, &DriverVideoCapture::error, m_stateError);

connect(m_stateAcquire, &QState::entered, this, &DriverVideoCapture::on_readFrame);

m_engine->setInitialState(m_stateIdle);
m_engine->start();

Callback of looping state_Acquire:

void DriverVideoCapture::on_readFrame()
{
    cv::Mat frame;
    if (!m_videoCapture->read(frame))
        throwError("failed to read frame");

    QCoreApplication::processEvents();

    emit frameReady(frame);
}