Displaying images using openGL and QT

saghi.sharif

Hi,

In my application , I have a pre-recorded image sequences which have been loaded to a CV::Mat variables. I want to show them in my application as fast as possible. What I'm doing now is to set every pixel of the cv::mat in qImage and using a qlabel I display them. But it's very slow and I want to try OpenGL instead to make it faster. I don't know where I have to begin, If any one of you have some suggestions or tutorials to share I would really appreciate it! :)

stereomatching

Are you copying the images from cv::Mat to QImage?
If you are copying, you could make QImage point to the memory cv::Mat point to

@template<typename T>
inline QImage reference_mat_to_qimage(cv::Mat &mat, QImage::Format format)
{
return QImage((T*)(mat.data), mat.cols, mat.rows, mat.step, format);
} @

If i am correct, what you want to do is showing the image store by cv::Mat by QLabel?
If that is the case, I have no idea how could the openGL help you boost up the performance
it would be reasonably if you try to use openGL to do some image processing jobs
but it make no sense if you want to send your data from cv::Mat to gpu and plot in onto the framebuffer

Besides, how do you know the performance bottleneck is QLabel?
Maybe the bottleneck is not QLabel?Do you use profiler to find out the bottleneck?

saghi.sharif

Thanks for your response :-)

What I'm doing now is to copy the image from cv::Mat to QImage and then set it to QLabel. The images are HDR and in EXR format and this is how I set them in QImage ;

@
int w = m_cvImage.cols;
int h = m_cvImage.rows;
int ch = m_cvImage.channels();

if (m_image->width() != w || m_image->height() != h)
{
delete m_image;
m_image = 0;
}

if (m_image == 0)
m_image = new QImage(m_cvImage.cols, m_cvImage.rows,
QImage::Format_ARGB32_Premultiplied);

w *= ch;
if (m_cvImage.isContinuous())
{
w *= h;
h = 1;
}

int isCol = ch >= 3;
for (int y = 0; y < h; y++)
{

const float* srcrow = (float*) m_cvImage.ptr(y);
QRgb destrow = (QRgb) m_image->scanLine(y);
int i = 0;
for (int x = 0; x < w; x += ch)
destrow[i++] = qRgb(std::min(255, std::max(0, int(255 * srcrow[x
+ 2 * isCol]))), std::min(255, std::max(0, int(255
* srcrow[x + isCol]))), std::min(255, std::max(0, int(255
* srcrow[x]))));
}
@

When I remove this part or crop the image the process is so much faster and it's reasonable since the size of image is 1654 * 1654. That's why I think this is the bottle neck of my application. I will do some image processing in the future that's the reason I was thinking about OpenGL. I tried your function but the image is not shown correctly. I hope I have explained my problem :-)

stereomatching

bq. I tried your function but the image is not shown correctly. I hope I have explained my problem

Maybe you are setting the wrong T and QImage::Format?
It is just an idea of how to transform between QImage and cv::Mat
you need to make sure their format and pointer type are match

Part of my libs

@
QImage mat_to_qimage_ref(cv::Mat &mat, int qimage_format)
{
typedef unsigned char UC;

switch(mat.type()){

case CV_8UC3 :
{        
    cv::cvtColor(mat, mat, CV_BGR2RGB);

    return reference_mat_to_qimage<UC>(mat, QImage::Format_RGB888);
}

case CV_8U :
{
    return reference_mat_to_qimage<UC>(mat, QImage::Format_Indexed8);
}

case CV_8UC4 :
{        
    if(qimage_format == -1){
      return reference_mat_to_qimage<UC>(mat, QImage::Format_ARGB32);
    }
    else{
      return reference_mat_to_qimage<UC>(mat, QImage::Format(qimage_format));
    }
}

default :
    return QImage();
}

}

@

I haven't finished all of the transformation between QImage and openCV yet
but this should cover most of the image types(atleast on desktop)

I implement 4 functions for transformation between QImage and openCV

@
mat_to_qimage_cpy
mat_to_qimage_ref
qimage_to_mat_cpy
qimage_to_mat_ref@

github link
"libs":https://github.com/stereomatchingkiss/qimage_and_mat

If there are any bugs or you find a better way to implement it, please tell me, I want to learn.

All of them are aim for desktop(without excessive test yet) since I haven't tried to deploy my apps
on mobile yet(waiting for Qt5.1).

A good reference of how to combine opengl and Qt5

"Qt5 + openGL":http://qt-project.org/wiki/Developer-Guides

Do remember that the prebuild Qt5 we download from the website only support the shader language
of openGL ES2.0(or openGL2.x?), if you need openGL of desktop, you have to build it by yourself if
I haven't make any mistakes(I am fine with it since I want to deploy my apps on mobile).

stereomatching

I forgot to mention one thing, if the QImage or cv::Mat after transfer need to reallocate their memory, you better copy the image rather than reference to them, because this may cause some undefined behaviour.