OpenCV VideoCapture Failure from QThread



  • Hi guys,

    I'm having a bit of a problem opening and reading video files using OpenCV's VideoCapture object from a QThread.

    If the two VideoCaptures are moved to the main GUI thread, I can use a QTimer to read frames every millisecond with no issues whatsoever.

    If placed in a QThread, I can open video files before the thread starts but once it is started, I get crashes with the message below:

    The program has unexpectedly finished.
    The process was ended forcefully.
    C:/Users/Roham/Documents/build-VidCap-Desktop_Qt_5_11_3_MinGW_32bit-Debug/debug/VidCap.exe crashed.

    The QThread run() function is written this way:

    while(true)
        {
            for(int i = 0; i < 2; i++)
            {
                if(!video[i].isOpened())
                {
                    video[i].release();
                }
                else
                {
                    if(!video[i].read(testMat[i]))
                    {
                        video[i].release();
                    }
                    else
                    {
                        cv::cvtColor(testMat[i],testMat[i],cv::COLOR_BGR2RGB);
                        imgOut[i] = QImage((uchar*)(testMat[i].data),testMat[i].cols,testMat[i].rows, QImage::Format_RGB888);
                        emit processedImage(i,imgOut[i]);
                    }
                }
            }
        }
    

    I use buttons from my form to trigger the file change event from the main GUI thread. The buttons end up calling this function in the QThread:

    void Test::load(const int &vid, const int &file)
    {
        switch(file)
        {
            case 0:
                video[vid].open(QDir::currentPath().toStdString()+"/vid0.mp4",cv::CAP_ANY);
                break;
            case 1:
                video[vid].open(QDir::currentPath().toStdString()+"/vid1.mp4",cv::CAP_ANY);
                break;
            case 2:
                video[vid].open(QDir::currentPath().toStdString()+"/vid2.mp4",cv::CAP_ANY);
                break;
        }
    }
    

    Is there something else I should be doing? Like stopping the thread temporarily when loading a new file? I've come across a couple of posts from others with the same issue, but it makes no sense to me why this would happen.

    Thanks!


  • Lifetime Qt Champion

    Hi,

    Are you subclassing QThread ?



  • @SGaist Yes I am. I've been told this is not the correct way to use QThreads but other say its just a matter of choice. Should I try passing a QObject to a thread instead?

    Its worth mentioning that the problem is narrowed down to the .read() method of VideoCapture. Loading doesn't cause a crash. I tried using a lock and mutex right before reading a frame which seems to avoid a crash but the .read() method still hangs. Using mutex alone (locked before read, unlocked after) causes a crash as before.


  • Moderators

    @rtavakko said in OpenCV VideoCapture Failure from QThread:

    I've been told this is not the correct way to use QThreads but other say its just a matter of choice.

    Subclassing QThreads is a valid approach, but you must do it correctly.

    Anyway, it sounds like you are calling VideoCapture methods from multiple threads. Is that class thread-safe? Do you know if you are allowed to call its methods from multiple threads?



  • @JKSH I override the run function of the subclassed QThread object. I only call VideoCapture methods from a single QThread. These aren't thread-safe from what I've read online but even so I shouldn't have an issue if accessing them from one thread only unless there's something going on in the background that I don't know.

    The issue is that if I move the VideoCaptures to the main thread, the issue is fixed but I get crashes when they are part of the QThread.


  • Moderators

    @rtavakko said in OpenCV VideoCapture Failure from QThread:

    I only call VideoCapture methods from a single QThread.

    Important note: Just because you call something from within a QThread method, that does not mean the code runs in the related thread!

    Earlier, you said

    The buttons end up calling this function in the QThread:

    Well, Test::load() is running in the main thread, not your other thread.

    If you want to verify this yourself, add qDebug() << QThread::currentThread() into the top of main(), the top of your run() function, and the top of Test::load(). This will show you which thread is running each function.

    Read the QThread documentation carefully: https://doc.qt.io/qt-5/qthread.html. It says, "...a developer who wishes to invoke slots in the new thread must use the worker-object approach; new slots should not be implemented directly into a subclassed QThread.

    When you reimplemented QThread::run(), you used an infinite loop (while (true)). This means your thread does not have an event loop, which means you cannot "trigger" anything in that thread.



  • @JKSH Thanks for the explanation. When a GUI button is pressed the thread is QThread(0x1922e9b0) and when the Test::load() is called, the thread is Test(0x197ded38) which is a bit confusing in terms of the names but looks like they are separate. I'm going to switch to what the documentation outlines and get back with the results.
    On this note, how do I re-implement the event loop or make the thread continuously run without overriding the run() functions?


  • Qt Champions 2018

    @rtavakko said in OpenCV VideoCapture Failure from QThread:

    On this note, how do I re-implement the event loop or make the thread continuously run without overriding the run() functions?

    See the docs: https://doc.qt.io/qt-5/qthread.html#run

    "... The default implementation simply calls exec()."



  • @JKSH I went back and noticed I had the qDebug() statement in the wrong place. You were correct about which thread a called method runs in. If the main thread calls a method in another thread, that method still runs in the main thread as the documentation says.

    The issue was related to thread-safety as you suggested. Its also important to make sure we are not reading from a VideoCapture device that is in the process of loading a new video.

    Here are my 'read' / 'load' methods for anyone having issues with this:

    void Test::load(const int &vid, const int &file)
    {
        QMutexLocker lock(&mutex);
    
        loading[vid] = true;
    
        switch(file)
        {
            case 0:
                loadFlag[vid] = video[vid].open(QDir::currentPath().toStdString()+"/vid0.mp4",cv::CAP_ANY);
                break;
            case 1:
                loadFlag[vid] = video[vid].open(QDir::currentPath().toStdString()+"/vid1.mp4",cv::CAP_ANY);
                break;
            case 2:
                loadFlag[vid] = video[vid].open(QDir::currentPath().toStdString()+"/vid2.mp4",cv::CAP_ANY);
                break;
        }
        loading[vid] = false;
    }
    
    bool Test::read(const int &vid)
    {
        loadFlag[vid] = video[vid].isOpened();
    
        if(!loadFlag)
        {
            video[vid].release();
            return false;
        }
        else
        {
            QMutexLocker locker(&mutex);
            readFlag[vid] = video[vid].read(readMat[vid]);
    
            if(!readFlag[vid])
            {
                video[vid].release();
                return readFlag[vid];
            }
            else
            {
                if(!readMat[vid].empty())
                {
                    cv::cvtColor(readMat[vid],readMat[vid],cv::COLOR_BGR2RGB);
                    imgOut[vid] = QImage(readMat[vid].data, readMat[vid].cols, readMat[vid].rows, QImage::Format_RGB888);
                    return true;
                }
            }
        }
    }
    

    Also the run method in the QThread object:

    void Test::run()
    {
        while(true)
        {
            for(int i = 0; i < 2; i++)
            {
                if(!loading[i])
                {
                    bool tmp = this->read(i);
                    if(tmp)
                    {
                        emit processedImage(i,imgOut[i]);
                    }
                    else
                    {
                        //SH*T HAPPENED
                    }
                }
            }
        }
    }
    

    I'm still a bit confused about where exec() would come in, the documentation is a little hazy to me. Any SAFE threads you guys could reference that clears up how I need to implement run() and exec() together?

    Thanks a lot for all of your help with this. Much appreciated!


  • Moderators

    @rtavakko said in OpenCV VideoCapture Failure from QThread:

    I'm still a bit confused about where exec() would come in, the documentation is a little hazy to me. Any SAFE threads you guys could reference that clears up how I need to implement run() and exec() together?

    The default implementation of QThread::run() is:

    void QThread::run() {
        this->exec(); // Starts an event loop in the thread
    }
    

    If you want to reimplement run(), you have 2 options:

    // Option 1: Run an event loop
    void MyEventLoopThread::run() {
        /* Add your custom initialization code here */
    
        this->exec(); // Starts an event loop in the thread
    }
    
    // Option 2: Don't run an event loop; manually write an infinite loop
    void MyPollingLoopThread::run() {
        /* Add your custom initialization code here */
    
        while ( !this->isInterruptionRequested() ) // Starts an infinite loop in the thread
        {
            /* Add your custom looping code here */
        }
    

    Notice that Option #1 is just like your main thread:

    int main(int argc, char **argv) {
        QApplication app(argc, argv);
    
        /* Add your custom initialization code here */
    
        app.exec(); // Starts an event loop in the main thread
    }
    

    Finally, Option #1 and Option #2 cannot be combined. Your thread can have either an event loop (exec()) OR a polling loop (while (true)). It cannot have both.

    • If you choose Option #1, your thread can receive signals.
    • If you choose Option #2, your thread cannot receive signals.

    Does this all make sense?



  • @JKSH Thanks a lot for taking the time to explain. The concept makes sense. After this->exec() is run with the event loop approach, the thread object is listed for deletion and its run() method cannot be called again. Is that correct? If so, then the thread object has to be created every time?


  • Moderators

    You're welcome.

    @rtavakko said in OpenCV VideoCapture Failure from QThread:

    After this->exec() is run with the event loop approach, the thread object is listed for deletion and its run() method cannot be called again. Is that correct? If so, then the thread object has to be created every time?

    Nope, exec() does not mark anything for deletion.

    You can certainly start() and quit() a thread with an event loop multiple times. (Note: Quit does nothing if your thread has no event loop)



  • @JKSH That makes sense. I'll give that a try. Thanks again!