Important: Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

Showing video in GUI does not work (black screen)



  • Hi there,

    I develop a Qt App that should show a given videoinput on the GUI. For this I use a third-party-library from Blackmagic.
    My program compiles, but when I start the video the frame output area remains black. For transfering frames I use the event loop and in a special VideoScreen class I create a instance of a thread.

    //VideoScreen.cpp
    VideoScreen::VideoScreen(QWidget* parent) :
    	m_previewHelper(nullptr)
    {
    	//Registering the type name of type CComPtr<IDeckLinkVideoFrame>, requires include<QMetaType>
    	//Create and destroy objects of the type dyamically at runtime after registration
    	qRegisterMetaType<CComPtr<IDeckLinkVideoFrame>>("CComPtr<IDeckLinkVideoFrame>");
    
    	m_vsThread = new QThread();
    	m_drawFrame = new VideoScreenHelper(parent);
    	m_drawFrame->moveToThread(m_vsThread);
    
    	QObject::connect(m_drawFrame, &VideoScreenHelper::FrameChanged, this, &VideoScreen::HandleFrame, Qt::QueuedConnection);
    	QObject::connect(m_vsThread, &QThread::finished, m_drawFrame, &QObject::deleteLater); //Deletes VideoScreenHelper object
    	QObject::connect(m_vsThread, &QThread::finished, m_vsThread, &QObject::deleteLater); //Deletes QThread object
    	m_vsThread->start();
    	qDebug() << "VideoScreen Ctor: Thread-ID is " << QThread::currentThreadId();
    }
    

    When I debug the program I see that the event loop has a different thread id than the function which emits a new frame.
    That is the function that emits the frame:

    //VideoScreenHelper.cpp
    HRESULT VideoScreenHelper::DrawFrame(IDeckLinkVideoFrame* frame) {
    	QMetaObject::invokeMethod(this, "FrameChanged", Qt::QueuedConnection, Q_ARG(CComPtr<IDeckLinkVideoFrame>, frame));
    	qDebug() << "VideoScreenHelper: emitting FrameChanged.";
    	return S_OK;
    }
    

    So my question is whats going wrong here? Or is there no problem with the threading? I study many tutorials (1, 2, 3, 4) but I'm not sure if my work is the right way.


  • Lifetime Qt Champion

    Hi,

    Decklink used to have an SDK with examples using Qt, is that not the case anymore ?

    If not, one of the possible solution would be to build a QImage in the callback and send it to be rendered.



  • @SGaist
    Hi,
    I know, there are examples in the SDK that uses Qt and the signal/slot mechanism. But they are not self-explanatory and I also get a black screen when I compile this examples (yes I have connected the camera and the capture device).



  • Not sure why you need a thread to send frames. I simply stream a video to a video sink of qtgstreamer without any thread.


  • Lifetime Qt Champion

    @makopo I remember having to dig a bit in the code but it was not that arcane.

    Since you do not get any image, the silly question is: did you check the configuration ?
    Did you validate that the acquisition is working with a tool like DaVinici Resolve ?



  • @JoeCFD

    I had read that connect and emit have to run in the same thread. That's why I use QThread.



  • @SGaist

    With BM Media Express I get an input stream. So I think the configuration should work.


  • Lifetime Qt Champion

    @makopo said in Showing video in GUI does not work (black screen):

    @JoeCFD

    I had read that connect and emit have to run in the same thread. That's why I use QThread.

    No, otherwise the worker object paradigm would not work. When letting the default connection type (AutoConnection) at emission time it will be decided whether it's a direct call or a queued call.

    From memory, Decklink uses a callback method so you do not need to have an additional thread running.

    Did you check the video frame format ? Is it RGB ? If not, do you convert it to something usable by Qt ?



  • @SGaist said in Showing video in GUI does not work (black screen):

    @makopo said in Showing video in GUI does not work (black screen):

    @JoeCFD

    I had read that connect and emit have to run in the same thread. That's why I use QThread.

    No, otherwise the worker object paradigm would not work. When letting the default connection type (AutoConnection) at emission time it will be decided whether it's a direct call or a queued call.

    Well, good to know...
    The display mode is 1080p25 in YUV. But BM also use this mode in the examples. Actually I get memory problems. Don't know if I have problems with pointers.


  • Lifetime Qt Champion

    What kind of memory problem ?



  • @SGaist

    It is difficult for me to retrace that, because actually the event-loop runs. I tried to work with raw pointers (just out of desperation), and with that I got a read access violation error with code 0xC0000005. The event loop runs for a short inconstant time and than the programm exits with this error. Between that, the debugger detects an c0000374 error. I think it was a buffer overflow because I didn't implement the deletetion of the raw-pointers.

    But actually there is no problem with pointers I think. As I can see in the console the program with the event loop runs (only the screen is black).

    VideoScreen: GetScreenPeviewCallback was called. TEST
    ControlVideo: ScreenPreviewCallback works.
    ControlVideo: SetCallback works.
    ControlVideo: Video input is enabled.
    ControlVideo: Stream has started.
    VideoScreen: GetScreenPeviewCallback was called. TEST
    QVideoMeter: VALUE PREVIEWCALLBACK  0x20ee0b32d00
    QVideoMeter: StartButton clicked. Stream is running.
    VideoScreenHelper: DrawFrame is called. Thread-ID is  0x3338
    VideoScreen: HandleFrame method is called. Thread-ID:  0x1628
    VideoScreenHelper: DrawFrame is called. Thread-ID is  0x3338
    VideoScreen: HandleFrame method is called. Thread-ID:  0x1628
    //and so on
    

    What strange is, I think, is that GetScreenPreviewCallback is called twice.



  • It looks like that there is a problem with the initialized object of the IDeckLinkGLScreenPreviewHelper, I create with CoCreateInstance() and the functions of the QOpenGLWidget class. I use QOpenGLWidget::initializeGL() and QOpenGLWidget()::paintGL to call equal named functions of IDeckLinkGLScreenPreviewHelper.

    //looks like "m_previewHelper" is null
    void VideoScreen::initializeGL() {
        if (m_previewHelper)  m_previewHelper->InitializeGL();
    }
    

    But when I debug the program I get the error message: 0xC0000005: Read access violation at position 0x0000000000000000. Looks like m_previewHelper points to null.
    Strangely enough, Handleframe (the slot function), which also uses the previewHelper object, is executed.

    //looks like m_previewHelper is not null
    void VideoScreen::HandleFrame(CComPtr<IDeckLinkVideoFrame> theFrame) {
        if (m_previewHelper) m_previewHelper->SetFrame(&(*theFrame));
    }
    

    I had written a function that that calls CoCreateInstance and create with this the previewHelper object. In the constructor I assign this function to the member variable. The variable has the same datatype than the function.

    //call in constructor
    m_previewHelper = ScreenPreviewHelperInstance();
    

    When I print the value of m_previewHelper I get something like 0x29e3d3a78d0 and so on. m_previewHelper seams to be not null.
    I did not understand that problem. Did someone of you have an idea whats going wrong?


  • Lifetime Qt Champion

    @makopo said in Showing video in GUI does not work (black screen):

    Did someone of you have an idea whats going wrong?

    A pointer != nullptr does not mean that it is valid. It can be that the memory it is pointing to was already released. Did you release m_previewHelper somewhere?



  • @jsulm
    I use CComPtrs and with that functions of the IUnknown interface. Also IUnknown::Release.


  • Lifetime Qt Champion

    Did you consider implementing my suggestion rather than moving theses pointers around ?

    I don't think there's a guarantee that the buffer they are pointing to will be valid when the callback method is done.



  • @SGaist

    Did you mean building a QImage in an own method instead of using the DrawFrame method from the callback interface?


  • Lifetime Qt Champion

    Build the QImage in the callback and send it further using invokeMethod.



  • @SGaist

    I found by accident that my program renders a single frame of the camera input, after I start the loop and minimize the GUI of the program. When I reopen the GUI a new frame is rendered and also, as I can see in the console, the paintGL() method of the QOpenGLWidget interface is called. I think the paintGL()method is not implemented well. I will take look to the OpenGL part.

    Do you agree that there is a problem with OpenGL? Or do you think the cause of this is the event loop?
    Sorry for all this silly questions. I'am not very experienced in programming and this is a studyproject from university. Some thinks sounds new to me.


  • Lifetime Qt Champion

    That QOpenGLWidget is new to me, there's no reference to it in your code.

    I would recommend minimizing your application so that you can concentrate on the Decklink integration part.



  • I checked the DeckLink intergration and I think there is no problem. I come bck to the OpenGl part and tried to draw a triangle with OpenGL in QOpenGLWidget::paintGL(). The function is called, but the graphic isn't rendered.

    void VideoScreen::paintGL() {
    	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
    	glBegin(GL_TRIANGLES);
    		glColor3f(1, 0, 0);
    		glVertex3f(-0.5, -0.5, 0);
    		glColor3f(0, 1, 0);
    		glVertex3f(0.5, -0.5, 0);
    		glColor3f(1, 0, 0);
    		glVertex3f(0.0, -0.5, 0);
    	glEnd();
    	qDebug() << "VideoScreen: paintGL works";
    }
    

    I also checked QOpenGLWidget::initializeGL() and the call of glClearColor works.

    void VideoScreen::initializeGL() {
    	initializeOpenGLFunctions();
    	glClearColor(1, 1, 0, 1); //after calling this the widget is yellow
    	m_previewHelper->InitializeGL();
    	qDebug() << "VideoScreen: initalizeGL works.";
    }
    

    The OpenGL context is created in the main-method.

    QSurfaceFormat format;
    format.setDepthBufferSize(24);
    format.setStencilBufferSize(8);
    format.setVersion(3, 2);
    format.setProfile(QSurfaceFormat::CoreProfile);
    QSurfaceFormat::setDefaultFormat(format);
    

    The QOpenGLWidget part is implemented in a own class.

    class OpenGLScreen : public QOpenGLWidget, protected QOpenGLFunctions {};
    

    I create a instance of the class which inherits from QMainWindow. Then the OpenGLScreen Widget is added to a layout:

    QScreen::QScreen(QWidget* parent)
        : QMainWindow(parent), 
        m_ui(new Ui::MainWindow)
    {
        m_screen = new OpenGLScreen(parent);
        m_horizontalScreen->addWidget(m_screen);
    
    QWidget::show();
    

    Thanks for a little hint whats going wrong here.


  • Lifetime Qt Champion

    I don't see any projection matrix initialization.

    In your code are you sure you are properly setting up the texture to be painted if using a texture for the video image ?



  • @SGaist

    Thanks for advice. I tried to initialize the projection matrix in QOpenGLWidget::resizeGL(int w, int h), but I also get an empty widget.

    void VideoScreen::resizeGL(int w, int h) {
        glViewport(0, 0, w, h);
        glMatrixMode(GL_PROJECTION);
        glLoadIdentity();
        glOrtho(-1.0, 1.0, -1.0, 1.0, -1.0, 1.0);
        glMatrixMode(GL_MODELVIEW);
        qDebug() << "VideoScreen: resizeGl works";
    }
    

    According to the decklink documention the texture should be generated with a special interface, called IDeckLinkScreenPreviewHelper.
    So in VideoScreen::paintGL() a call of m_screenPreviewHelper->PaintGL() should render Texture and Shader. But it is not deeply descripted in the documentation. Also in concerning sample code Shaders and Textures are not used.
    When I include m_screenPreviewHelper->PaintGL() I get a black widget. After minimizing the main window paintGL is called and after reopening the window too. With this I get a single (freezed) frame rendered.


  • Lifetime Qt Champion

    Can you provide a link to the actual Decklink example you are using as reference ?



  • @SGaist

    You can finde the sample code here: QuadPreview. The SDK is not shared with github e.g. , so I uploaded it to private cloud.

    In this Qt example the matrix identity and the perspective are set with the QMatrix4x4 class. Is this modern way for initializing a matrix? Or is there no difference between

    glViewport(0, 0, w, h);
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    glOrtho(-1.0, 1.0, -1.0, 1.0, -1.0, 1.0);
    glMatrixMode(GL_MODELVIEW);
    

    and

    m_proj.setToIdentity();
    m_proj.perspective(45.0f, GLfloat(w) / h, 0.01f, 100.0f);
    


  • I fixed the issue with calling QWidget::update() in QOpenGLWidget::paintGL(). Now the videostream works.


  • Lifetime Qt Champion

    Glad you found a solution and thanks for sharing !

    Note that this is a rather unexpected solution has you should not need to do that.



  • @SGaist said in Showing video in GUI does not work (black screen):

    Note that this is a rather unexpected solution has you should not need to do that.

    That sounds unsatisfactory. In fact signal an slot runs in the same thread.
    In another source I found that signal and connect should run in the same thread. In my code these are different.


  • Lifetime Qt Champion

    Signals and slots are not required to be handled in objects belonging to the same thread.


Log in to reply