Continuous image capturing in the background for a WEC7 device using QT 4.7.4

[MagnusMessi]

I am currently trying to capture images continuously for a WEC7+ARMv4I device by using QTimer and QPixmap. The issue is that, when image capturing is started the UI becomes slower to use. The class where image capturing is taking place does make use of QThread to run it on a seperate thread.

My question is what more can I do to keep UI running as normal (i.e keep it responsive)? I am happy to share the code I am using if required.

[SGaist]

Hi and welcome to devnet,

Where are these images coming from ?
What does your GUI do ?
What size are these images ?
Where are you writing them to ?
What format are you using ?

[MagnusMessi]

Hi SGaist,

Thank you for your prompt reply and apologies for my delayed reply.

1. The images are captured from the UI while browsing the UI (i.e. real-time capturing).
2. GUI is for a printer, so you can imagine it is doing a lot of things in the background.
3. The image size is 640*480.
4. I used to write them to QBuffer, but now I am just doing QPixmap->grabwindow(...) to write and capture images from the UI.
5. When I was using QBuffer I had it in "PNG" format, but with QPixmap I am not specifying any image format as it is not required?

One thing to note is that there are a lot of backend calls take place at any give time, these include QT code calls + backend code calls. I am trying to capture at least 5 images per second through an interval timer, desired is 10.

[SGaist]

That makes things clearer.

QPixmap does not work with file format but with a format suitable for GUI representation on hardware.

Can you explain why you need that capture for ? It might be possible to find a better way of doing this than rendering the widgets every 200ms.

[MagnusMessi]

@SGaist

I would like to see if I can create "gif" on this embedded device. The approach I am taking is capturing a certain number of images continuously while browsing the UI and then I am using a third part library to create gif by providing the captured images. On win32 application (apologies for not mentioningthis earlier), it is working way better, its just that on this embedded device it is becoming a challenge to do so (even after applying some threading concepts).

[SGaist]

Depending on your hardware, you should use an image format that requires less compression (but you will need more space).

Can you show your grabbing pipeline ?

[MagnusMessi]

Below is the sample code:

void A::captureScreenShot()
{
   QMutexLocker locker(&m_mutex);
   screenShotCount++;
   if(screenShotCount <= 25)
   {
       //option 1
       //QSize snapSize(640, 480);
       //QImage out(snapSize, QImage::Format_RGB16);
       //QPainter tap(&out);
       //tap.setRenderHints( QPainter::Antialiasing | QPainter::SmoothPixmapTransform);
       //render(&tap);

       //option 2
       //QPixmap pixmap = QPixmap::grabWindow(QApplication::activeWindow()->winId());
   }
   else
   {
      m_timer->stop();
   }
}

CaptureScreenShot() is a slot which gets executed based on timer interval (currently I have set it to 200 milliseconds). The timer is started from another function, which is associated with a QML file.

I have tried both options above yet I still see lagging UI while browsing (third option not shown above was using option 1 + QBuffer and writing image data into the buffer with "png" format). It feels like the issue is more to do with not applying multi-threading than image capturing, although I do run the whole class and the timer in a separate thread.

[SGaist]

Using QImage is a good idea to offload the work to QThread. How are you doing your threading ?

[MagnusMessi]

It is as below:

A::A()
{
    m_timer = new QTimer(this);
    m_screenShotHandlerThread = new QThread(this);
    moveToThread(m_screenShotHandlerThread);
    m_timer->moveToThread(m_screenShotHandlerThread); //Not sure if required

    m_screenShotHandlerThread->start();
}

A::~A()
{
    delete m_timer;     
    m_screenShotHandlerThread->quit();     
    m_screenShotHandlerThread->wait();
}

The class is instantiated at start up of the software, thus it will also start/run the thread. The timer is started based on a long press on a certain widget of the software.

While searching for background image processing, I came across this site where multi-threading is applied for a mobile device software. It looks extensive and nicely described however, I am not sure how much of it can be applied in my scenario.

[SGaist]

Are you using a queue to process your images ?

[MagnusMessi]

No, I am not doing that. Would like to know more about it. Also let me know if I can improve threading in any way.

[SGaist]

Do you mean your thread handling is moving the timer to a different thread ?

[MagnusMessi]

No its not, its just that if threading needs improvement in any way like only starting the thread after the timer starts or so. If not then I will leave it as it is and focus on queues.

[SGaist]

One problem might come from where things are done. The capture can only happen in the GUI thread (usually the main thread). All the rest can happen in the other thread(s) but the implementation must be properly done. Depending on how you did your stuff, you might still be doing the work in the main thread.

[MagnusMessi]

I had assumed that instantiating a new thread would allow the captures to take place outside the main thread, which would keep GUI responsive. However if it is not then obviously I would require something like queues as you have mentioned earlier.

[MagnusMessi]

@SGaist it would be great if you can provide a link for this queues approach you asked me to look into it, as I have not been able to find anything concrete.

[SGaist]

QQueue

Note that you still have to properly protect its access when reading and writing.

What threading experience do you have ?

[MagnusMessi]

@SGaist thanks for this. Well not much threading experience as I just get to learn about it while working on this project.

[SGaist]

Then please take the time to read the QThread documentation thoroughly. Use the Qt 5 documentation as it covers way more strategies than the Qt 4 version with better examples. Just keep in mind the small differences between the two versions.

[MagnusMessi]

@SGaist sure, will do :)