[SOLVED] QT MainWindow freezes when closing OpenCV Webcam Window - new QT Window outside MainWindow



  • Hi,

    I am working with VS2008, QT Designer, QT 4.8.5, OpenCV and PS3EyeSDK.

    I have a QT Application with a MainWindow that opens an OpenCV Window of a Webcam via the connection of a PushButton.

    This works well, I also have a slot stopCam() that closes down the Camera Window, but when I hit it it does close but the MainWindow freezes and nothing can be done anymore.

    Is there anything to be concerned of about threading?

    Anything I'm doing wrong ? Here are some snippets:

    qtdevc.cpp - main program

    @//get ID of participant
    int qtDEVC::startCam()
    {
    //qt part

    //ui.startCam->setText("Hi!");
    QString ID;
    //get qString Participant Number
    ID = ui.lineEditID->text();
    //convert to standard string
    std::string IDString = ID.toLocal8Bit().constData();
    //qDebug()<<ID;
    ui.lineEditID->setDisabled(true);
    ui.pushButton_startCam->setDisabled(true);
    ...
    ...
    ...

    for(int i = 0; i < numCams; i++)
    {
    printf("Stopping capture on camera %d\n", i+1);
    cam[i]->StopCapture();
    delete cam[i];
    }

    return 0;
    //return (IDString);
    }

    void qtDEVC::stopCam()
    {
    qDebug()<<("stopCam()");
    cam[0]->StopCapture();
    delete cam[0];
    ui.lineEditID->setDisabled(false);
    ui.pushButton_startCam->setDisabled(false);
    }@

    CLEyeCameraCapture.cpp

    @ bool CLEyeCameraCapture::StartCapture(string ID)
    {
    _running = true;
    _participant = ID;
    cvNamedWindow(_windowName, CV_WINDOW_AUTOSIZE);
    // Start CLEye image capture thread
    _hThread = CreateThread(NULL, 0, &CLEyeCameraCapture::CaptureThread, this, 0, 0);
    if(_hThread == NULL)
    {
    MessageBoxA(NULL,"Could not create capture thread","CLEyeMulticamTest", MB_ICONEXCLAMATION);
    return false;
    }
    return true;
    }

    void CLEyeCameraCapture::StopCapture()
    {
    if(!_running) return;
    _running = false;
    WaitForSingleObject(_hThread, 1000);
    cvDestroyWindow(_windowName);
    }@


  • Lifetime Qt Champion

    Hi,

    IIRC OpenCV uses Qt for it's highgui module. You might have some bad interaction between your application and this module.



  • Sounds great... how to fix it ? I really need to get this to work.

    switch to opencv only ? can i do this ?


  • Lifetime Qt Champion

    It depends on your use of opencv, you can also do the preview with Qt



  • ok that seems fine, either switch completely to qt or open cv highgui if i understand correctly.

    if i switch to qt i basically have to exchange
    cvNamedWindow(_windowName, CV_WINDOW_AUTOSIZE)
    ??


  • Lifetime Qt Champion

    Depending on how you get the image, it's format and the frame rate, you can use a QLabel and set a QPixmap on it, OpenGL etc...

    You could also check with the OpenCV guys to see if there's a possibility to integrate highgui with a Qt application



  • hm.. i made a new form now called camWindow which is based on QWidget and it opens fine, now i see if it closes and if I can transfer an IPL Image to QT somehow reliably fast



  • OK here we go...some good news but mostly akward stuff i don't really understand:

    I managed to create a new window and all necessary files.

    If i want to open and close this window within my main Q:MainWindow application i have no problem doing so. My problem is that I have other Code coming from PS3 Eye SDK that I poorly tried to migrate into this QT app for my purpose.

    And within this class i want to access the new window to open up and show a IPL Image that i converted to QImage, doing this compiles ok but when I try to open a new Camera Window ( @_camWindow = new CameraWindow(); @) which i defined an instance of this CLEyeCameraCapture Class i get an errorC2248.

    I assume this is because of my poor understanding in C++ and my guess is i have to migrate this class somehow in QT but I really struggle finding the right constructors and formal code to do so.

    Maybe some of you might see the problem very quick or can give me some advice. Here's the code

    CLEyeCameraCapture.h

    #pragma once
    #include <camerawindow.h>

    @// Sample camera capture class from CL Eye SDK edited to fit my needs
    class CLEyeCameraCapture
    {
    CHAR _windowName[256];
    GUID _cameraGUID;
    CLEyeCameraInstance _cam;
    CLEyeCameraColorMode _mode;
    CLEyeCameraResolution _resolution;
    float _fps;
    HANDLE _hThread;
    bool _running;
    std::string _participant;
    CameraWindow _camWindow;
    public:
    CLEyeCameraCapture(LPSTR windowName, GUID cameraGUID, CLEyeCameraColorMode mode, CLEyeCameraResolution resolution, float fps) :
    _cameraGUID(cameraGUID), _cam(NULL), _mode(mode), _resolution(resolution), _fps(fps), _running(false)
    {
    strcpy(_windowName, windowName);
    }

    CLEyeCameraCapture()
    {
    }

    double GetRandomNormalized();
    bool StartCapture(std::string ID);
    void StopCapture();
    void IncrementCameraParameter(int param);
    void DecrementCameraParameter(int param);
    void Run();
    static DWORD WINAPI CaptureThread(LPVOID instance);
    static QImage IplImage2QImage(const IplImage *iplImage);
    };
    @

    CLEyeCameraCapture.cpp
    @
    #include <stdafx.h>
    #include <iostream>

    #include <QtGui>
    #include <QtGui/QApplication>
    #include <QtGui/QImage>
    #include <QVector>
    #include <QWidget>

    #include <camerawindow.h>
    #include <CLEyeCameraCapture.h>

    using namespace std;

    //I don't know how exactly to define the constructor here so to work with QT and also I think i have to do a different definition in the header file?

    ...
    //start Capturing with appropriate ID
    bool CLEyeCameraCapture::StartCapture(string ID)
    {
    _camWindow = new CameraWindow();
    _running = true;
    _participant = ID;
    //I try to make this window obsolete and exchange with class CameraWindow : public QWidget
    cvNamedWindow(_windowName, CV_WINDOW_AUTOSIZE);
    // Start CLEye image capture thread
    _hThread = CreateThread(NULL, 0, &CLEyeCameraCapture::CaptureThread, this, 0, 0);
    if(_hThread == NULL)
    {
    MessageBoxA(NULL,"Could not create capture thread","CLEyeMulticamTest", MB_ICONEXCLAMATION);
    return false;
    }
    return true;
    }
    ...
    @



  • Maybe it is more easy to rephrase my question:

    I have a main window for my main application. When i click a button I want to create a camera class object that runs a loop and opens a window to see the images.

    What would be the common structure for that?


  • Lifetime Qt Champion

    It depends on how do you get the images from this SDK.

    You could use a timer to retrieve the images and update a QLabel for example.



  • I get my images from a run loop within this CLEyeCameraCapture class in a defined fps rate. I would like to parse the signal of this runloop to any QT Window eg. pseudo:

    @run(){
    while(runnig){

    iplimage = getImage(Camera);
    QImage = IPLtoQimage(iplimage);
    newwindow.show(QImage);
    }
    }@

    I am thinking now that the cleanest way to do that would be to create a newWindow in the mainWindow .cpp and then access this window in my QObject class (which inherits the run loop and camera access).

    But i lack of experience to find the right expressions.


  • Lifetime Qt Champion

    Then I would recommend taking a look at the "MandelBrot example":http://qt-project.org/doc/qt-4.8/threads-mandelbrot.html



  • hey SGaist, thanks for the info but i already got the whole loop and my problem is just to connect the image created in the loop with the QDialog (parent) of the class.

    I took a look at this:

    http://www.sobbayi.com/blog/software-development/access-members-parent-class-qt-c/

    which kind of is what I'm searching. it works ok except i cannot see the image due to i made something wrong with the label.

    in my child class this compiles fine:

    @ while(_running)
    {
    CLEyeCameraGetFrame(_cam, pCapBuffer);

    // Resize Image to fit screen size
    cvResize(pCapImage,resizedpCapImage,CV_INTER_NN);
    cvShowImage(_windowName, resizedpCapImage);

    *qimg = IplImage2QImage(resizedpCapImage);

    parentqlabel->setPixmap(QPixmap::fromImage(*qimg));
    parentqlabel->show();@

    but does not show the image


  • Lifetime Qt Champion

    The technique described in this article is exactly what is discouraged to do in general. The basic idea with Qt is to emit a signal from the child class that the parent class connects to. So the child class doesn't need to know anything about the parent (keeps the code clean). Or if you don't want a signal (in a dialog case) you use getters once your done with the dialog.

    Back to your problem, you are using an infinite loop in the main thread so this will block everything, hence the mandlebrot example.

    Also why make qimg a pointer ? You don't need that, just make in a regular QImage. You will avoid memory management problem.



  • I get that its discouraged and i get that this is not pro's programming here but I need to get it to work asap so I'm willing to take hate :)

    In my CameraCapture class i declared the variable like the following, if you can point me out the correct way it would be nice. As I mentioned I am a C++ starter.

    @ QImage *qimg = new QImage();

    //Create Buffer for pCap
    PBYTE pCapBuffer = NULL;

    // Create camera instance
    _cam = CLEyeCreateCamera(_cameraGUID, _mode, _resolution, _fps);
    if(_cam == NULL)
    ...
    ...
    ...

    // Start capturing
    CLEyeCameraStart(_cam);
    cvGetImageRawData(pCapImage, &pCapBuffer);
    // image capturing loop
    while(_running)
    {
    CLEyeCameraGetFrame(_cam, pCapBuffer);

    // Resize Image to fit screen size
    cvResize(pCapImage,resizedpCapImage,CV_INTER_NN);
    cvShowImage(_windowName, resizedpCapImage);

    *qimg = IplImage2QImage(resizedpCapImage);@

    Also, as you complained the child process and parent process might be switched here, so I guess I could make the CaptureClass parent of the CameraWindow Class which would leed to:

    MainWindow (QMainWindow) --> creates CaptureObject (QObject)
    CaptureObject (QObject) -> creates and is parent of CameraWindow (QDialog)

    This is the only solution I can think of as I see that I have thread problems because I can not create communication between CaptureObject and Camera Window now.

    @ parentqlabel->setTextFormat(Qt::RichText);
    parentqlabel->setText("textest");@

    creates

    ASSERT failure in QCoreApplication::sendEvent: "Cannot send events to objects owned by a different thread. Current thread 48cb8f8. Receiver '' (of type 'QLabel') was created in thread 23c0790", file kernel\qcoreapplication.cpp, line 535
    ASSERT failure in QCoreApplication::sendEvent: "Cannot send events to objects owned by a different thread. Current thread 23c0790. Receiver '' (of type 'QTextDocument') was created in thread 48cb8f8", file kernel\qcoreapplication.cpp, line 535

    This is because the CameraCaptureSDK provided code creates a thread :

    @_hThread = CreateThread(NULL, 0, &CLEyeCameraCapture::CaptureThread, this, 0, 0);@

    ...
    @
    DWORD WINAPI CLEyeCameraCapture::CaptureThread(LPVOID instance)
    {
    // seed the rng with current tick count and thread id
    srand(GetTickCount() + GetCurrentThreadId());
    // forward thread to Capture function
    CLEyeCameraCapture *pThis = (CLEyeCameraCapture *)instance;
    pThis->Run();
    return 0;@



  • ok so i put my new window as part of the cameracapture class and it opens and closes...wonderfull, but MainWindow GUI freezes again when hitting the close button on it to close my window.

    here i am..back at the beginning


  • Lifetime Qt Champion

    That's exactly why I suggested the mandelbrot example, you have there an example of how to implement an forever loop and interact with your application.

    You can't update GUI elements from outside the main thread.

    Could you post a link to the sdk ?



  • Here are the links:

    http://codelaboratories.com/research/view/cl-eye-platform-cpp-sample

    http://codelaboratories.com/get/cl-eye-sdk/

    the window now closes (typing mistake) correctly. (yeij)

    so now if i do this:

    @CameraWindow::CameraWindow(QWidget *parent)
    : QDialog(parent)
    {
    ui.setupUi(this);
    imageLabel = new QLabel(this);
    imageLabel->setTextFormat(Qt::RichText);
    imageLabel->setText("textest");
    }

    CameraWindow::~CameraWindow()
    {

    }
    @

    and create a window within my capture class this opens correctly and text is displayed. but if i try to make this a function

    @CameraWindow::CameraWindow(QWidget *parent)
    : QDialog(parent)
    {
    ui.setupUi(this);
    imageLabel = new QLabel(this);
    }

    CameraWindow::~CameraWindow()
    {

    }

    void CameraWindow::dispText(){
    imageLabel->setTextFormat(Qt::RichText);
    imageLabel->setText("textest");
    }@

    and call camerawindow->dispText() from my capture class i get a "Cannot create children for a parent that is in a different thread" which is akward because obviously imageLabel should be an instance within my CameraWindow not anything else. as CameraWindow is created within this class i really wonder what's going on :)

    To your remark: Even if i create a dialog within another thread (obviously another thread because it says so) the GUI Element QLabel i create within this new dialog can only be updated in my MainWindow thread?

    If I understand you pointing to the mandelbrot example the solution would be to exchange the orignal sdk thread:

    @HANDLE _hThread@

    @_hThread = CreateThread(NULL, 0, &CLEyeCameraCapture::CaptureThread, this, 0, 0);@

    with another QT Thread creation...

    @
    RenderThread::RenderThread(QObject *parent)
    : QThread(parent)

    @

    which is kind of what i do not feel capable of :)


  • Lifetime Qt Champion

    Creating widgets outside the main thread is also forbidden.

    Here is a rough idea on how to use a subclassed QThread to get the data from the camera and send them to your widget. (Just the run function actually but it's the most important)

    @
    void MyQThread::run()
    int w, h;
    IplImage *pCapImage;
    PBYTE pCapBuffer = NULL;

        // Create camera instance
        _cam = CLEyeCreateCamera(_cameraGUID, _mode, _resolution, _fps);
        if(_cam == NULL)        return;
        // Get camera frame dimensions
        CLEyeCameraGetFrameDimensions(_cam, w, h);
    

    // Depending on color mode chosen, create the appropriate OpenCV image
    if(_mode == CLEYE_COLOR)
    pCapImage = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 4);
    else
    pCapImage = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 1);

        // Set some camera parameters
        CLEyeSetCameraParameter(_cam, CLEYE_GAIN, 20);
        CLEyeSetCameraParameter(_cam, CLEYE_EXPOSURE, 511);
        CLEyeSetCameraParameter(_cam, CLEYE_ZOOM, (int)(GetRandomNormalized()*100.0));
        CLEyeSetCameraParameter(_cam, CLEYE_ROTATION, (int)(GetRandomNormalized()*300.0));
    
        // Start capturing
        CLEyeCameraStart(_cam);
        // image capturing loop
        while(_running)
        {
            cvGetImageRawData(pCapImage, &pCapBuffer);
            CLEyeCameraGetFrame(_cam, pCapBuffer);
             emit capturedImage(IplImage2QImage(pCapImage));
        }
        // Stop camera capture
        CLEyeCameraStop(_cam);
        // Destroy camera object
        CLEyeDestroyCamera(_cam);
        _cam = NULL;
    }
    

    @

    Depending on the image format you could even directly capture to a QImage avoiding the conversion.

    Hope it helps



  • thank you so much for your answers and your advice.
    i'll try to implement this with looking at the mandelbrot example and post updates here.



  • I followed the mandelbrot example and did alike with my class.

    Now, I am not able to acces wait(), start(), isRunning() or any other static (protected?) function which seems either not to be declared or not found.

    I included the following, something's missing ?
    @
    #include <QCoreApplication>
    #include <QThread>
    #include <QObject>
    #include <QMutex>
    #include <QWaitCondition>
    @

    ok...it seems that this is because i create a dialog window within the render thread that seems not to be able to parent correctly.

    i guess the way would be to open the camerwindow dialog within the mainwindow class and connect the signal of the renderthread to the camerwindow ?


  • Lifetime Qt Champion

    Did you subclass QThread and use the code in the run function ?

    No widget stuff whatsoever in another thread than the GUI thread (they mean it)

    That would be one way yes



  • Ok, seems that something is wrong with the connection:

    @1>.\camerawindow.cpp(14) : error C2664: 'bool QObject::connect(const QObject *,const char *,const QObject *,const char *,Qt::ConnectionType)': Konvertierung des Parameters 1 von 'RenderThread **' in 'const QObject *' nicht möglich
    1> Die Typen, auf die verwiesen wird, sind nicht verknüpft; die Konvertierung erfordert einen reinterpret_cast-Operator oder eine Typumwandlung im C- oder Funktionsformat.@

    although i defined RenderThread as in the example..

    @class RenderThread : public QThread
    {
    Q_OBJECT
    @


  • Lifetime Qt Champion

    it seems that you are giving the address of the pointer to your RenderThread.

    Do you have something like that :

    @RenderThread *renderThread = new RenderThread;
    conncet(&renderThread, etc....)@

    ?



  • thanx, i'm tryin hard to get this to work ;)
    it's really chaos now..


  • Lifetime Qt Champion

    The best is to go one step at a time, first get the data from the camera, then add the widget etc... Trying to make all at the same time is recipe for failure



  • going with your version didn't work out so i went with this:

    camerawindow.cpp:

    @CameraWindow::CameraWindow(QWidget *parent)
    : QDialog(parent)
    {
    ui.setupUi(this);
    QLabel *imageLabel = new QLabel(this);

    qRegisterMetaType<QImage>("QImage");
    connect(&thread, SIGNAL(renderedImage(QImage)),
    this, SLOT(updatePixmap(QImage)));

    setWindowTitle(tr("multithread"));
    }

    ...

    int CameraWindow::startCaptureThread(){
    this->show();

    RenderThread *thread = new RenderThread(this, windowName, guid, CLEYE_COLOR_RAW, CLEYE_VGA, 50);

    thread->StartCapture("abc");
    }
    @

    which seems to work fine as looping the run() while loop correctly,

    but obviously the updatePixmap function is never called so the objects might be mislinked or created in wrong syntax... frustrating ;)



  • i tried with another connection:

    @ connect(&thread, SIGNAL(sendText(std::string)),
    this, SLOT(debugSomething(std::string)));@

    @void CameraWindow::debugSomething(std::string something){
    QString somethings(something.c_str());
    qDebug()<<somethings;
    }@

    renderthread.cpp:
    @while(_running)
    {
    emit sendText(text);
    }@

    and this as well seems not to emit anything or not correctly connect to the qdialog. why is this not working as it should ?


  • Lifetime Qt Champion

    You should replace all std::string with QString, that will avoid you some problems.
    Your while loop might be a bit short and might block the main thread.

    If your code is not top secret, could you share it i.e. via gitorious/github/other vcs service ? That might make things easier to look at



  • Hi SGaist,

    after getting a bit tired refitting the Mandelbrot Example to my needs I asked this question at StackOverflow. It turns out that there are some issues the way the example uses threads. I don't know if I could have saved my code but I was getting depressed so I recoded the way somebody pointed me out at StackOverflow and this worked instantly and also avoids image conversion as i can use the opencv namendWindow.

    @//create new thread
    QThread* thread = new QThread;

    //init camera with guid, raw color capture, vga format and 50fps
    cameraCapture = new CLEyeCameraCapture(winName, folder, userID, guid, CLEYE_COLOR_RAW, CLEYE_VGA, 50);
    cameraCapture->moveToThread(thread);

    connect(thread, SIGNAL(started()), cameraCapture, SLOT(StartCapture()));
    connect(cameraCapture, SIGNAL(finished()), thread, SLOT(quit()));
    connect(cameraCapture, SIGNAL(finished()), cameraCapture, SLOT(deleteLater()));
    connect(thread, SIGNAL(finished()), thread, SLOT(deleteLater()));
    connect(ui.pushButton_startLogging, SIGNAL ( clicked() ),this,SLOT( activateLogging() ) );
    connect(ui.pushButton_stopLogging, SIGNAL ( clicked() ),this,SLOT( deactivateLogging() ) );
    @

    I tried to follow this "example.":http://mayaposch.wordpress.com/2011/11/01/how-to-really-truly-use-qthreads-the-full-explanation/

    Here is the "Question on SO.":http://stackoverflow.com/questions/18594528/create-thread-in-qdialog-and-emit-signals-to-qdialog-in-qt

    Thank you for your help. Now i avoided the freezes. It might be of interest for people who using any conversion from IplImage to QImage: non of the conversions are fast enough for realtime (I did not measure wether the conversion or the displaying method is slowing down the process).

    Thank you anyway for your help. I guess it is wise for anybody who rebuilds this to use their original code and put it in a Worker which then gets moved to a QThread.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.