Continuous image stream QThread

Guess11

Hi, my application continiuosly grab images in a while loop in the thread, and then emiting them to main thread. The main problem is that everythings goes well I can display image in separate window(not qt), if thread doesn't emit images. If thread starts to emit images camera start to lag and discarded some images.

Camera thread looks as follow:

class CameraObject : public QObject
{
    Q_OBJECT
public:
    explicit CameraObject()
    { // init camera, setup camera parameters 
     ...}

    ~CameraObject();

public slots:
    Q_INVOKABLE void StateChanged(const bool &newState)
    {state = newState;}

    void ContinuousShot();

signals:
    Q_INVOKABLE void NewImage(const QImage &im);

private:
    //bool variable to check state
    // start and stop stream
    bool state;
    // camera object
    Camera_t *camera;
    QMutex mutex;
    int64_t width = 2048;
    int64_t height = 2048;
};

Method ContinuousShot:

void  CameraObject : : ContinuousShot()   
{
try
    {
        // Create a pylon image.
        CPylonImage PylonImage;
        //camera -> Open();
        if( !camera->IsGrabbing())
        {
            camera->StartGrabbing(GrabStrategy_OneByOne);
        }
        // pointer to results
        while( camera->IsGrabbing() )
        {
            // smart pointer to grab results
            Pylon::CBaslerUsbGrabResultPtr  ptrGrabResult;

            if( !state )
            {
              camera->StopGrabbing();
            }
            else
            {
                // retrieving results from camera to smart pointer
                if( camera->RetrieveResult(waitTime, ptrGrabResult, 
                                       TimeoutHandling_ThrowException))
                {
                    // pylon image
                    Pylon::CPylonImage image;

                    cout<< "Number of skiped images"<<endl;
                    cout<< ptrGrabResult->GetNumberOfSkippedImages()<<endl;

                    if( ptrGrabResult.IsValid() && ptrGrabResult->GrabSucceeded())
                    {
                        // from smart pointer to image
                        image.AttachGrabResultBuffer( ptrGrabResult);
                        // converting to QImage
                        mutex.lock();
                        QImage qimage;
                        // grayscale image
                        qimage = QImage((uchar*) image.GetBuffer(), width, height, 
                                                                       QImage::Format_Indexed8);
                        //Pylon::DisplayImage(1,  image);
                        // emiting New Image
                        emit NewImage(qimage);
                        mutex.unlock();
                      }
                }
                else
                {
                    cout<<("Results can't be retrived.");
                }
            // end of else
            }
        // end of while
        }
    // end of try   
    }
    catch(GenICam::GenericException &e)
    {
       cout<<(e.GetDescription())<<endl;
    }
}

The main Thread (main window)

namespace Ui {
class ImageWindow;
}

class ImageWindow : public QMainWindow
{
    Q_OBJECT

public:
    explicit ImageWindow(QWidget *parent = 0) :QMainWindow(parent),
    ui(new Ui::ImageWindow)
{
    // camera
    camera = new CameraObject;
    camera->moveToThread( &CameraThread);

    // handle results
    connect(camera, &CameraObject::NewImage,
            this, &ImageWindow::NewImage,Qt::QueuedConnection);

    // star cContinuousshoot
    connect(this, &ImageWindow::StartContinuousShot,
            camera, &CameraObject::ContinuousShot,Qt::QueuedConnection);

    // state changed
    connect(this, &ImageWindow::StateChanged,
            camera, &CameraObject::StateChanged,Qt::QueuedConnection);

    connect(&CameraThread, &QThread::finished,
           camera, &QObject::deleteLater);

     CameraThread.start();
}

    ~ImageWindow();
private slots:

    void on_StartButton_clicked()
    {
     emit StateChanged(true);
     emit StartContinuousShot();
    }

    void on_StopButton_clicked()
    {
     emit StateChanged(false);
    }

    void NewImage(const QImage &image)
    {
     qDebug()<<"New Image";
     Q_UNUSED(image);
    }

signals:
   // user click button
   void StateChanged(const bool &click);
   // StartStream
   void StartContinuousShot();

private:
    Ui::ImageWindow *ui;
    // Camera obj
    CameraObject *camera;
    QThread CameraThread;
};

I will be glad to any suggestions or to any help. Sorry for so long piece of code.

SGaist

Hi,

You should benchmark your code to see where you are losing time.

Guess11

Hi, after testing. I get that converting time is from 0.015 - 0.02 miliseconds, showing - from 1.45 to 1.5 miliseconds and emiting is from 0.04 to 0.05 miliseconds, retriving the results is from 0.02 miliseconds to 26 miliseconds(due to the documentation of camera, thread will wait until waitTime expirece or new data appear). However if I will comment display line, camera still will discard some data.

void CameraObject::ContinuousShot()
{
...
t = (double)getTickCount();
if( camera->RetrieveResult(waitTime, ptrGrabResult, 
                                       TimeoutHandling_ThrowException)){

t = 1000*((double)getTickCount() - t)/getTickFrequency();
cout<<"Time to retrive results: "<<t<<" miliseconds"<<endl;
cout<<"\n";
...

t = (double)getTickCount();
qimage = QImage((uchar*)image.GetBuffer(),
                                  width,
                                   height,
                                   QImage::Format_Indexed8);
t = 1000*((double)getTickCount() - t)/getTickFrequency();
cout<<"Time to convert image: "<<t<<" miliseconds"<<endl;
cout<<"\n";

t = (double)getTickCount();
Pylon::DisplayImage(1, image);
t = 1000*((double)getTickCount() - t)/getTickFrequency();
cout<<"Time to show image: "<<t<<" miliseconds"<<endl;
cout<<"\n";


// emiting New Image
qDebug()<<"In emmitting id thread "<<QThread::currentThreadId();
 
t = (double)getTickCount();
emit NewImage(qimage);
t = 1000*((double)getTickCount() - t)/getTickFrequency();
cout<<"Time emit image: "<<t<<" miliseconds"<<endl;
cout<<"\n";
}

SGaist

Isn't there a callback alternative that doesn't need to block the thread ?

Guess11

Unfortunately the most expensive operations Camera.Open() and Camera.Close happens in the constructor and destructor, in addition camera is a singleton so it can be only one instance of camera object. Thus I don't see approach to do it in calback way. However, after deleting macro Q_INVOKABLE, camera do not discard images anymore. But still I don't know the reason why it was happening. Thank you for the replies and help.

SGaist

I missed the fact that you put a Q_INVOKABLE in front of your signal. I don't know what kind of impact it can have.