Showing video in GUI does not work (black screen)

makopo

Hi there,

I develop a Qt App that should show a given videoinput on the GUI. For this I use a third-party-library from Blackmagic.
My program compiles, but when I start the video the frame output area remains black. For transfering frames I use the event loop and in a special VideoScreen class I create a instance of a thread.

//VideoScreen.cpp
VideoScreen::VideoScreen(QWidget* parent) :
	m_previewHelper(nullptr)
{
	//Registering the type name of type CComPtr<IDeckLinkVideoFrame>, requires include<QMetaType>
	//Create and destroy objects of the type dyamically at runtime after registration
	qRegisterMetaType<CComPtr<IDeckLinkVideoFrame>>("CComPtr<IDeckLinkVideoFrame>");

	m_vsThread = new QThread();
	m_drawFrame = new VideoScreenHelper(parent);
	m_drawFrame->moveToThread(m_vsThread);

	QObject::connect(m_drawFrame, &VideoScreenHelper::FrameChanged, this, &VideoScreen::HandleFrame, Qt::QueuedConnection);
	QObject::connect(m_vsThread, &QThread::finished, m_drawFrame, &QObject::deleteLater); //Deletes VideoScreenHelper object
	QObject::connect(m_vsThread, &QThread::finished, m_vsThread, &QObject::deleteLater); //Deletes QThread object
	m_vsThread->start();
	qDebug() << "VideoScreen Ctor: Thread-ID is " << QThread::currentThreadId();
}

When I debug the program I see that the event loop has a different thread id than the function which emits a new frame.
That is the function that emits the frame:

//VideoScreenHelper.cpp
HRESULT VideoScreenHelper::DrawFrame(IDeckLinkVideoFrame* frame) {
	QMetaObject::invokeMethod(this, "FrameChanged", Qt::QueuedConnection, Q_ARG(CComPtr<IDeckLinkVideoFrame>, frame));
	qDebug() << "VideoScreenHelper: emitting FrameChanged.";
	return S_OK;
}

So my question is whats going wrong here? Or is there no problem with the threading? I study many tutorials (1, 2, 3, 4) but I'm not sure if my work is the right way.

SGaist

Hi,

Decklink used to have an SDK with examples using Qt, is that not the case anymore ?

If not, one of the possible solution would be to build a QImage in the callback and send it to be rendered.

makopo

@SGaist
Hi,
I know, there are examples in the SDK that uses Qt and the signal/slot mechanism. But they are not self-explanatory and I also get a black screen when I compile this examples (yes I have connected the camera and the capture device).

JoeCFD

Not sure why you need a thread to send frames. I simply stream a video to a video sink of qtgstreamer without any thread.

SGaist

@makopo I remember having to dig a bit in the code but it was not that arcane.

Since you do not get any image, the silly question is: did you check the configuration ?
Did you validate that the acquisition is working with a tool like DaVinici Resolve ?

makopo

@JoeCFD

I had read that connect and emit have to run in the same thread. That's why I use QThread.

makopo

@SGaist

With BM Media Express I get an input stream. So I think the configuration should work.

SGaist

@makopo said in Showing video in GUI does not work (black screen):

@JoeCFD

I had read that connect and emit have to run in the same thread. That's why I use QThread.

No, otherwise the worker object paradigm would not work. When letting the default connection type (AutoConnection) at emission time it will be decided whether it's a direct call or a queued call.

From memory, Decklink uses a callback method so you do not need to have an additional thread running.

Did you check the video frame format ? Is it RGB ? If not, do you convert it to something usable by Qt ?

makopo

@SGaist said in Showing video in GUI does not work (black screen):

@makopo said in Showing video in GUI does not work (black screen):

@JoeCFD

I had read that connect and emit have to run in the same thread. That's why I use QThread.

No, otherwise the worker object paradigm would not work. When letting the default connection type (AutoConnection) at emission time it will be decided whether it's a direct call or a queued call.

Well, good to know...
The display mode is 1080p25 in YUV. But BM also use this mode in the examples. Actually I get memory problems. Don't know if I have problems with pointers.

SGaist

What kind of memory problem ?

makopo

@SGaist

It is difficult for me to retrace that, because actually the event-loop runs. I tried to work with raw pointers (just out of desperation), and with that I got a read access violation error with code 0xC0000005. The event loop runs for a short inconstant time and than the programm exits with this error. Between that, the debugger detects an c0000374 error. I think it was a buffer overflow because I didn't implement the deletetion of the raw-pointers.

But actually there is no problem with pointers I think. As I can see in the console the program with the event loop runs (only the screen is black).

VideoScreen: GetScreenPeviewCallback was called. TEST
ControlVideo: ScreenPreviewCallback works.
ControlVideo: SetCallback works.
ControlVideo: Video input is enabled.
ControlVideo: Stream has started.
VideoScreen: GetScreenPeviewCallback was called. TEST
QVideoMeter: VALUE PREVIEWCALLBACK  0x20ee0b32d00
QVideoMeter: StartButton clicked. Stream is running.
VideoScreenHelper: DrawFrame is called. Thread-ID is  0x3338
VideoScreen: HandleFrame method is called. Thread-ID:  0x1628
VideoScreenHelper: DrawFrame is called. Thread-ID is  0x3338
VideoScreen: HandleFrame method is called. Thread-ID:  0x1628
//and so on

What strange is, I think, is that GetScreenPreviewCallback is called twice.

makopo

It looks like that there is a problem with the initialized object of the IDeckLinkGLScreenPreviewHelper, I create with CoCreateInstance() and the functions of the QOpenGLWidget class. I use QOpenGLWidget::initializeGL() and QOpenGLWidget()::paintGL to call equal named functions of IDeckLinkGLScreenPreviewHelper.

//looks like "m_previewHelper" is null
void VideoScreen::initializeGL() {
    if (m_previewHelper)  m_previewHelper->InitializeGL();
}

But when I debug the program I get the error message: 0xC0000005: Read access violation at position 0x0000000000000000. Looks like m_previewHelper points to null.
Strangely enough, Handleframe (the slot function), which also uses the previewHelper object, is executed.

//looks like m_previewHelper is not null
void VideoScreen::HandleFrame(CComPtr<IDeckLinkVideoFrame> theFrame) {
    if (m_previewHelper) m_previewHelper->SetFrame(&(*theFrame));
}

I had written a function that that calls CoCreateInstance and create with this the previewHelper object. In the constructor I assign this function to the member variable. The variable has the same datatype than the function.

//call in constructor
m_previewHelper = ScreenPreviewHelperInstance();

When I print the value of m_previewHelper I get something like 0x29e3d3a78d0 and so on. m_previewHelper seams to be not null.
I did not understand that problem. Did someone of you have an idea whats going wrong?

jsulm

@makopo said in Showing video in GUI does not work (black screen):

Did someone of you have an idea whats going wrong?

A pointer != nullptr does not mean that it is valid. It can be that the memory it is pointing to was already released. Did you release m_previewHelper somewhere?

makopo

@jsulm
I use CComPtrs and with that functions of the IUnknown interface. Also IUnknown::Release.

SGaist

Did you consider implementing my suggestion rather than moving theses pointers around ?

I don't think there's a guarantee that the buffer they are pointing to will be valid when the callback method is done.

makopo

@SGaist

Did you mean building a QImage in an own method instead of using the DrawFrame method from the callback interface?

SGaist

Build the QImage in the callback and send it further using invokeMethod.

makopo

@SGaist

I found by accident that my program renders a single frame of the camera input, after I start the loop and minimize the GUI of the program. When I reopen the GUI a new frame is rendered and also, as I can see in the console, the paintGL() method of the QOpenGLWidget interface is called. I think the paintGL()method is not implemented well. I will take look to the OpenGL part.

Do you agree that there is a problem with OpenGL? Or do you think the cause of this is the event loop?
Sorry for all this silly questions. I'am not very experienced in programming and this is a studyproject from university. Some thinks sounds new to me.

SGaist

That QOpenGLWidget is new to me, there's no reference to it in your code.

I would recommend minimizing your application so that you can concentrate on the Decklink integration part.

makopo

I checked the DeckLink intergration and I think there is no problem. I come bck to the OpenGl part and tried to draw a triangle with OpenGL in QOpenGLWidget::paintGL(). The function is called, but the graphic isn't rendered.

void VideoScreen::paintGL() {
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
	glBegin(GL_TRIANGLES);
		glColor3f(1, 0, 0);
		glVertex3f(-0.5, -0.5, 0);
		glColor3f(0, 1, 0);
		glVertex3f(0.5, -0.5, 0);
		glColor3f(1, 0, 0);
		glVertex3f(0.0, -0.5, 0);
	glEnd();
	qDebug() << "VideoScreen: paintGL works";
}

I also checked QOpenGLWidget::initializeGL() and the call of glClearColor works.

void VideoScreen::initializeGL() {
	initializeOpenGLFunctions();
	glClearColor(1, 1, 0, 1); //after calling this the widget is yellow
	m_previewHelper->InitializeGL();
	qDebug() << "VideoScreen: initalizeGL works.";
}

The OpenGL context is created in the main-method.

QSurfaceFormat format;
format.setDepthBufferSize(24);
format.setStencilBufferSize(8);
format.setVersion(3, 2);
format.setProfile(QSurfaceFormat::CoreProfile);
QSurfaceFormat::setDefaultFormat(format);

The QOpenGLWidget part is implemented in a own class.

class OpenGLScreen : public QOpenGLWidget, protected QOpenGLFunctions {};

I create a instance of the class which inherits from QMainWindow. Then the OpenGLScreen Widget is added to a layout:

QScreen::QScreen(QWidget* parent)
    : QMainWindow(parent), 
    m_ui(new Ui::MainWindow)
{
    m_screen = new OpenGLScreen(parent);
    m_horizontalScreen->addWidget(m_screen);

QWidget::show();

Thanks for a little hint whats going wrong here.