[SOLVED] QT MainWindow freezes when closing OpenCV Webcam Window - new QT Window outside MainWindow

[refuzee]

Hi,

I am working with VS2008, QT Designer, QT 4.8.5, OpenCV and PS3EyeSDK.

I have a QT Application with a MainWindow that opens an OpenCV Window of a Webcam via the connection of a PushButton.

This works well, I also have a slot stopCam() that closes down the Camera Window, but when I hit it it does close but the MainWindow freezes and nothing can be done anymore.

Is there anything to be concerned of about threading?

Anything I'm doing wrong ? Here are some snippets:

qtdevc.cpp - main program

@//get ID of participant
int qtDEVC::startCam()
{
//qt part

//ui.startCam->setText("Hi!");
QString ID;
//get qString Participant Number
ID = ui.lineEditID->text();
//convert to standard string
std::string IDString = ID.toLocal8Bit().constData();
//qDebug()<<ID;
ui.lineEditID->setDisabled(true);
ui.pushButton_startCam->setDisabled(true);
...
...
...

for(int i = 0; i < numCams; i++)
{
printf("Stopping capture on camera %d\n", i+1);
cam[i]->StopCapture();
delete cam[i];
}

return 0;
//return (IDString);
}

void qtDEVC::stopCam()
{
qDebug()<<("stopCam()");
cam[0]->StopCapture();
delete cam[0];
ui.lineEditID->setDisabled(false);
ui.pushButton_startCam->setDisabled(false);
}@

CLEyeCameraCapture.cpp

@ bool CLEyeCameraCapture::StartCapture(string ID)
{
_running = true;
_participant = ID;
cvNamedWindow(_windowName, CV_WINDOW_AUTOSIZE);
// Start CLEye image capture thread
_hThread = CreateThread(NULL, 0, &CLEyeCameraCapture::CaptureThread, this, 0, 0);
if(_hThread == NULL)
{
MessageBoxA(NULL,"Could not create capture thread","CLEyeMulticamTest", MB_ICONEXCLAMATION);
return false;
}
return true;
}

void CLEyeCameraCapture::StopCapture()
{
if(!_running) return;
_running = false;
WaitForSingleObject(_hThread, 1000);
cvDestroyWindow(_windowName);
}@

[SGaist]

Hi,

IIRC OpenCV uses Qt for it's highgui module. You might have some bad interaction between your application and this module.

[refuzee]

Sounds great... how to fix it ? I really need to get this to work.

switch to opencv only ? can i do this ?

[SGaist]

It depends on your use of opencv, you can also do the preview with Qt

[refuzee]

ok that seems fine, either switch completely to qt or open cv highgui if i understand correctly.

if i switch to qt i basically have to exchange
cvNamedWindow(_windowName, CV_WINDOW_AUTOSIZE)
??

[SGaist]

Depending on how you get the image, it's format and the frame rate, you can use a QLabel and set a QPixmap on it, OpenGL etc...

You could also check with the OpenCV guys to see if there's a possibility to integrate highgui with a Qt application

[refuzee]

hm.. i made a new form now called camWindow which is based on QWidget and it opens fine, now i see if it closes and if I can transfer an IPL Image to QT somehow reliably fast

[refuzee]

OK here we go...some good news but mostly akward stuff i don't really understand:

I managed to create a new window and all necessary files.

If i want to open and close this window within my main Q:MainWindow application i have no problem doing so. My problem is that I have other Code coming from PS3 Eye SDK that I poorly tried to migrate into this QT app for my purpose.

And within this class i want to access the new window to open up and show a IPL Image that i converted to QImage, doing this compiles ok but when I try to open a new Camera Window ( @_camWindow = new CameraWindow(); @) which i defined an instance of this CLEyeCameraCapture Class i get an errorC2248.

I assume this is because of my poor understanding in C++ and my guess is i have to migrate this class somehow in QT but I really struggle finding the right constructors and formal code to do so.

Maybe some of you might see the problem very quick or can give me some advice. Here's the code

CLEyeCameraCapture.h

#pragma once
#include <camerawindow.h>

@// Sample camera capture class from CL Eye SDK edited to fit my needs
class CLEyeCameraCapture
{
CHAR _windowName[256];
GUID _cameraGUID;
CLEyeCameraInstance _cam;
CLEyeCameraColorMode _mode;
CLEyeCameraResolution _resolution;
float _fps;
HANDLE _hThread;
bool _running;
std::string _participant;
CameraWindow _camWindow;
public:
CLEyeCameraCapture(LPSTR windowName, GUID cameraGUID, CLEyeCameraColorMode mode, CLEyeCameraResolution resolution, float fps) :
_cameraGUID(cameraGUID), _cam(NULL), _mode(mode), _resolution(resolution), _fps(fps), _running(false)
{
strcpy(_windowName, windowName);
}

CLEyeCameraCapture()
{
}

double GetRandomNormalized();
bool StartCapture(std::string ID);
void StopCapture();
void IncrementCameraParameter(int param);
void DecrementCameraParameter(int param);
void Run();
static DWORD WINAPI CaptureThread(LPVOID instance);
static QImage IplImage2QImage(const IplImage *iplImage);
};
@

CLEyeCameraCapture.cpp
@
#include <stdafx.h>
#include <iostream>

#include <QtGui>
#include <QtGui/QApplication>
#include <QtGui/QImage>
#include <QVector>
#include <QWidget>

#include <camerawindow.h>
#include <CLEyeCameraCapture.h>

using namespace std;

//I don't know how exactly to define the constructor here so to work with QT and also I think i have to do a different definition in the header file?

...
//start Capturing with appropriate ID
bool CLEyeCameraCapture::StartCapture(string ID)
{
_camWindow = new CameraWindow();
_running = true;
_participant = ID;
//I try to make this window obsolete and exchange with class CameraWindow : public QWidget
cvNamedWindow(_windowName, CV_WINDOW_AUTOSIZE);
// Start CLEye image capture thread
_hThread = CreateThread(NULL, 0, &CLEyeCameraCapture::CaptureThread, this, 0, 0);
if(_hThread == NULL)
{
MessageBoxA(NULL,"Could not create capture thread","CLEyeMulticamTest", MB_ICONEXCLAMATION);
return false;
}
return true;
}
...
@

[refuzee]

Maybe it is more easy to rephrase my question:

I have a main window for my main application. When i click a button I want to create a camera class object that runs a loop and opens a window to see the images.

What would be the common structure for that?

[SGaist]

It depends on how do you get the images from this SDK.

You could use a timer to retrieve the images and update a QLabel for example.

[refuzee]

I get my images from a run loop within this CLEyeCameraCapture class in a defined fps rate. I would like to parse the signal of this runloop to any QT Window eg. pseudo:

@run(){
while(runnig){

iplimage = getImage(Camera);
QImage = IPLtoQimage(iplimage);
newwindow.show(QImage);
}
}@

I am thinking now that the cleanest way to do that would be to create a newWindow in the mainWindow .cpp and then access this window in my QObject class (which inherits the run loop and camera access).

But i lack of experience to find the right expressions.

[SGaist]

Then I would recommend taking a look at the "MandelBrot example":http://qt-project.org/doc/qt-4.8/threads-mandelbrot.html

[refuzee]

hey SGaist, thanks for the info but i already got the whole loop and my problem is just to connect the image created in the loop with the QDialog (parent) of the class.

I took a look at this:

http://www.sobbayi.com/blog/software-development/access-members-parent-class-qt-c/

which kind of is what I'm searching. it works ok except i cannot see the image due to i made something wrong with the label.

in my child class this compiles fine:

@ while(_running)
{
CLEyeCameraGetFrame(_cam, pCapBuffer);

// Resize Image to fit screen size
cvResize(pCapImage,resizedpCapImage,CV_INTER_NN);
cvShowImage(_windowName, resizedpCapImage);

*qimg = IplImage2QImage(resizedpCapImage);

parentqlabel->setPixmap(QPixmap::fromImage(*qimg));
parentqlabel->show();@

but does not show the image

[SGaist]

The technique described in this article is exactly what is discouraged to do in general. The basic idea with Qt is to emit a signal from the child class that the parent class connects to. So the child class doesn't need to know anything about the parent (keeps the code clean). Or if you don't want a signal (in a dialog case) you use getters once your done with the dialog.

Back to your problem, you are using an infinite loop in the main thread so this will block everything, hence the mandlebrot example.

Also why make qimg a pointer ? You don't need that, just make in a regular QImage. You will avoid memory management problem.

[refuzee]

I get that its discouraged and i get that this is not pro's programming here but I need to get it to work asap so I'm willing to take hate :)

In my CameraCapture class i declared the variable like the following, if you can point me out the correct way it would be nice. As I mentioned I am a C++ starter.

@ QImage *qimg = new QImage();

//Create Buffer for pCap
PBYTE pCapBuffer = NULL;

// Create camera instance
_cam = CLEyeCreateCamera(_cameraGUID, _mode, _resolution, _fps);
if(_cam == NULL)
...
...
...

// Start capturing
CLEyeCameraStart(_cam);
cvGetImageRawData(pCapImage, &pCapBuffer);
// image capturing loop
while(_running)
{
CLEyeCameraGetFrame(_cam, pCapBuffer);

// Resize Image to fit screen size
cvResize(pCapImage,resizedpCapImage,CV_INTER_NN);
cvShowImage(_windowName, resizedpCapImage);

*qimg = IplImage2QImage(resizedpCapImage);@

Also, as you complained the child process and parent process might be switched here, so I guess I could make the CaptureClass parent of the CameraWindow Class which would leed to:

MainWindow (QMainWindow) --> creates CaptureObject (QObject)
CaptureObject (QObject) -> creates and is parent of CameraWindow (QDialog)

This is the only solution I can think of as I see that I have thread problems because I can not create communication between CaptureObject and Camera Window now.

@ parentqlabel->setTextFormat(Qt::RichText);
parentqlabel->setText("textest");@

creates

ASSERT failure in QCoreApplication::sendEvent: "Cannot send events to objects owned by a different thread. Current thread 48cb8f8. Receiver '' (of type 'QLabel') was created in thread 23c0790", file kernel\qcoreapplication.cpp, line 535
ASSERT failure in QCoreApplication::sendEvent: "Cannot send events to objects owned by a different thread. Current thread 23c0790. Receiver '' (of type 'QTextDocument') was created in thread 48cb8f8", file kernel\qcoreapplication.cpp, line 535

This is because the CameraCaptureSDK provided code creates a thread :

@_hThread = CreateThread(NULL, 0, &CLEyeCameraCapture::CaptureThread, this, 0, 0);@

...
@
DWORD WINAPI CLEyeCameraCapture::CaptureThread(LPVOID instance)
{
// seed the rng with current tick count and thread id
srand(GetTickCount() + GetCurrentThreadId());
// forward thread to Capture function
CLEyeCameraCapture *pThis = (CLEyeCameraCapture *)instance;
pThis->Run();
return 0;@

[refuzee]

ok so i put my new window as part of the cameracapture class and it opens and closes...wonderfull, but MainWindow GUI freezes again when hitting the close button on it to close my window.

here i am..back at the beginning

[SGaist]

That's exactly why I suggested the mandelbrot example, you have there an example of how to implement an forever loop and interact with your application.

You can't update GUI elements from outside the main thread.

Could you post a link to the sdk ?

[refuzee]

Here are the links:

http://codelaboratories.com/research/view/cl-eye-platform-cpp-sample

http://codelaboratories.com/get/cl-eye-sdk/

the window now closes (typing mistake) correctly. (yeij)

so now if i do this:

@CameraWindow::CameraWindow(QWidget *parent)
: QDialog(parent)
{
ui.setupUi(this);
imageLabel = new QLabel(this);
imageLabel->setTextFormat(Qt::RichText);
imageLabel->setText("textest");
}

CameraWindow::~CameraWindow()
{

}
@

and create a window within my capture class this opens correctly and text is displayed. but if i try to make this a function

@CameraWindow::CameraWindow(QWidget *parent)
: QDialog(parent)
{
ui.setupUi(this);
imageLabel = new QLabel(this);
}

CameraWindow::~CameraWindow()
{

}

void CameraWindow::dispText(){
imageLabel->setTextFormat(Qt::RichText);
imageLabel->setText("textest");
}@

and call camerawindow->dispText() from my capture class i get a "Cannot create children for a parent that is in a different thread" which is akward because obviously imageLabel should be an instance within my CameraWindow not anything else. as CameraWindow is created within this class i really wonder what's going on :)

To your remark: Even if i create a dialog within another thread (obviously another thread because it says so) the GUI Element QLabel i create within this new dialog can only be updated in my MainWindow thread?

If I understand you pointing to the mandelbrot example the solution would be to exchange the orignal sdk thread:

@HANDLE _hThread@

@_hThread = CreateThread(NULL, 0, &CLEyeCameraCapture::CaptureThread, this, 0, 0);@

with another QT Thread creation...

@
RenderThread::RenderThread(QObject *parent)
: QThread(parent)

@

which is kind of what i do not feel capable of :)

[SGaist]

Creating widgets outside the main thread is also forbidden.

Here is a rough idea on how to use a subclassed QThread to get the data from the camera and send them to your widget. (Just the run function actually but it's the most important)

@
void MyQThread::run()
int w, h;
IplImage *pCapImage;
PBYTE pCapBuffer = NULL;

    // Create camera instance
    _cam = CLEyeCreateCamera(_cameraGUID, _mode, _resolution, _fps);
    if(_cam == NULL)        return;
    // Get camera frame dimensions
    CLEyeCameraGetFrameDimensions(_cam, w, h);

// Depending on color mode chosen, create the appropriate OpenCV image
if(_mode == CLEYE_COLOR)
pCapImage = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 4);
else
pCapImage = cvCreateImage(cvSize(w, h), IPL_DEPTH_8U, 1);

    // Set some camera parameters
    CLEyeSetCameraParameter(_cam, CLEYE_GAIN, 20);
    CLEyeSetCameraParameter(_cam, CLEYE_EXPOSURE, 511);
    CLEyeSetCameraParameter(_cam, CLEYE_ZOOM, (int)(GetRandomNormalized()*100.0));
    CLEyeSetCameraParameter(_cam, CLEYE_ROTATION, (int)(GetRandomNormalized()*300.0));

    // Start capturing
    CLEyeCameraStart(_cam);
    // image capturing loop
    while(_running)
    {
        cvGetImageRawData(pCapImage, &pCapBuffer);
        CLEyeCameraGetFrame(_cam, pCapBuffer);
         emit capturedImage(IplImage2QImage(pCapImage));
    }
    // Stop camera capture
    CLEyeCameraStop(_cam);
    // Destroy camera object
    CLEyeDestroyCamera(_cam);
    _cam = NULL;
}

@

Depending on the image format you could even directly capture to a QImage avoiding the conversion.

Hope it helps

[refuzee]

thank you so much for your answers and your advice.
i'll try to implement this with looking at the mandelbrot example and post updates here.