Showing video in GUI does not work (black screen)

makopo

With BM Media Express I get an input stream. So I think the configuration should work.

SGaist

@makopo said in Showing video in GUI does not work (black screen):

@JoeCFD

I had read that connect and emit have to run in the same thread. That's why I use QThread.

No, otherwise the worker object paradigm would not work. When letting the default connection type (AutoConnection) at emission time it will be decided whether it's a direct call or a queued call.

From memory, Decklink uses a callback method so you do not need to have an additional thread running.

Did you check the video frame format ? Is it RGB ? If not, do you convert it to something usable by Qt ?

makopo

@SGaist said in Showing video in GUI does not work (black screen):

@makopo said in Showing video in GUI does not work (black screen):

@JoeCFD

I had read that connect and emit have to run in the same thread. That's why I use QThread.

No, otherwise the worker object paradigm would not work. When letting the default connection type (AutoConnection) at emission time it will be decided whether it's a direct call or a queued call.

Well, good to know...
The display mode is 1080p25 in YUV. But BM also use this mode in the examples. Actually I get memory problems. Don't know if I have problems with pointers.

SGaist

What kind of memory problem ?

makopo

@SGaist

It is difficult for me to retrace that, because actually the event-loop runs. I tried to work with raw pointers (just out of desperation), and with that I got a read access violation error with code 0xC0000005. The event loop runs for a short inconstant time and than the programm exits with this error. Between that, the debugger detects an c0000374 error. I think it was a buffer overflow because I didn't implement the deletetion of the raw-pointers.

But actually there is no problem with pointers I think. As I can see in the console the program with the event loop runs (only the screen is black).

VideoScreen: GetScreenPeviewCallback was called. TEST
ControlVideo: ScreenPreviewCallback works.
ControlVideo: SetCallback works.
ControlVideo: Video input is enabled.
ControlVideo: Stream has started.
VideoScreen: GetScreenPeviewCallback was called. TEST
QVideoMeter: VALUE PREVIEWCALLBACK  0x20ee0b32d00
QVideoMeter: StartButton clicked. Stream is running.
VideoScreenHelper: DrawFrame is called. Thread-ID is  0x3338
VideoScreen: HandleFrame method is called. Thread-ID:  0x1628
VideoScreenHelper: DrawFrame is called. Thread-ID is  0x3338
VideoScreen: HandleFrame method is called. Thread-ID:  0x1628
//and so on

What strange is, I think, is that GetScreenPreviewCallback is called twice.

makopo

It looks like that there is a problem with the initialized object of the IDeckLinkGLScreenPreviewHelper, I create with CoCreateInstance() and the functions of the QOpenGLWidget class. I use QOpenGLWidget::initializeGL() and QOpenGLWidget()::paintGL to call equal named functions of IDeckLinkGLScreenPreviewHelper.

//looks like "m_previewHelper" is null
void VideoScreen::initializeGL() {
    if (m_previewHelper)  m_previewHelper->InitializeGL();
}

But when I debug the program I get the error message: 0xC0000005: Read access violation at position 0x0000000000000000. Looks like m_previewHelper points to null.
Strangely enough, Handleframe (the slot function), which also uses the previewHelper object, is executed.

//looks like m_previewHelper is not null
void VideoScreen::HandleFrame(CComPtr<IDeckLinkVideoFrame> theFrame) {
    if (m_previewHelper) m_previewHelper->SetFrame(&(*theFrame));
}

I had written a function that that calls CoCreateInstance and create with this the previewHelper object. In the constructor I assign this function to the member variable. The variable has the same datatype than the function.

//call in constructor
m_previewHelper = ScreenPreviewHelperInstance();

When I print the value of m_previewHelper I get something like 0x29e3d3a78d0 and so on. m_previewHelper seams to be not null.
I did not understand that problem. Did someone of you have an idea whats going wrong?

jsulm

@makopo said in Showing video in GUI does not work (black screen):

Did someone of you have an idea whats going wrong?

A pointer != nullptr does not mean that it is valid. It can be that the memory it is pointing to was already released. Did you release m_previewHelper somewhere?

makopo

@jsulm
I use CComPtrs and with that functions of the IUnknown interface. Also IUnknown::Release.

SGaist

Did you consider implementing my suggestion rather than moving theses pointers around ?

I don't think there's a guarantee that the buffer they are pointing to will be valid when the callback method is done.

makopo

@SGaist

Did you mean building a QImage in an own method instead of using the DrawFrame method from the callback interface?

SGaist

Build the QImage in the callback and send it further using invokeMethod.

makopo

@SGaist

I found by accident that my program renders a single frame of the camera input, after I start the loop and minimize the GUI of the program. When I reopen the GUI a new frame is rendered and also, as I can see in the console, the paintGL() method of the QOpenGLWidget interface is called. I think the paintGL()method is not implemented well. I will take look to the OpenGL part.

Do you agree that there is a problem with OpenGL? Or do you think the cause of this is the event loop?
Sorry for all this silly questions. I'am not very experienced in programming and this is a studyproject from university. Some thinks sounds new to me.

SGaist

That QOpenGLWidget is new to me, there's no reference to it in your code.

I would recommend minimizing your application so that you can concentrate on the Decklink integration part.

makopo

I checked the DeckLink intergration and I think there is no problem. I come bck to the OpenGl part and tried to draw a triangle with OpenGL in QOpenGLWidget::paintGL(). The function is called, but the graphic isn't rendered.

void VideoScreen::paintGL() {
	glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
	glBegin(GL_TRIANGLES);
		glColor3f(1, 0, 0);
		glVertex3f(-0.5, -0.5, 0);
		glColor3f(0, 1, 0);
		glVertex3f(0.5, -0.5, 0);
		glColor3f(1, 0, 0);
		glVertex3f(0.0, -0.5, 0);
	glEnd();
	qDebug() << "VideoScreen: paintGL works";
}

I also checked QOpenGLWidget::initializeGL() and the call of glClearColor works.

void VideoScreen::initializeGL() {
	initializeOpenGLFunctions();
	glClearColor(1, 1, 0, 1); //after calling this the widget is yellow
	m_previewHelper->InitializeGL();
	qDebug() << "VideoScreen: initalizeGL works.";
}

The OpenGL context is created in the main-method.

QSurfaceFormat format;
format.setDepthBufferSize(24);
format.setStencilBufferSize(8);
format.setVersion(3, 2);
format.setProfile(QSurfaceFormat::CoreProfile);
QSurfaceFormat::setDefaultFormat(format);

The QOpenGLWidget part is implemented in a own class.

class OpenGLScreen : public QOpenGLWidget, protected QOpenGLFunctions {};

I create a instance of the class which inherits from QMainWindow. Then the OpenGLScreen Widget is added to a layout:

QScreen::QScreen(QWidget* parent)
    : QMainWindow(parent), 
    m_ui(new Ui::MainWindow)
{
    m_screen = new OpenGLScreen(parent);
    m_horizontalScreen->addWidget(m_screen);

QWidget::show();

Thanks for a little hint whats going wrong here.

SGaist

I don't see any projection matrix initialization.

In your code are you sure you are properly setting up the texture to be painted if using a texture for the video image ?

makopo

@SGaist

Thanks for advice. I tried to initialize the projection matrix in QOpenGLWidget::resizeGL(int w, int h), but I also get an empty widget.

void VideoScreen::resizeGL(int w, int h) {
    glViewport(0, 0, w, h);
    glMatrixMode(GL_PROJECTION);
    glLoadIdentity();
    glOrtho(-1.0, 1.0, -1.0, 1.0, -1.0, 1.0);
    glMatrixMode(GL_MODELVIEW);
    qDebug() << "VideoScreen: resizeGl works";
}

According to the decklink documention the texture should be generated with a special interface, called IDeckLinkScreenPreviewHelper.
So in VideoScreen::paintGL() a call of m_screenPreviewHelper->PaintGL() should render Texture and Shader. But it is not deeply descripted in the documentation. Also in concerning sample code Shaders and Textures are not used.
When I include m_screenPreviewHelper->PaintGL() I get a black widget. After minimizing the main window paintGL is called and after reopening the window too. With this I get a single (freezed) frame rendered.

SGaist

Can you provide a link to the actual Decklink example you are using as reference ?

makopo

@SGaist

You can finde the sample code here: QuadPreview. The SDK is not shared with github e.g. , so I uploaded it to private cloud.

In this Qt example the matrix identity and the perspective are set with the QMatrix4x4 class. Is this modern way for initializing a matrix? Or is there no difference between

glViewport(0, 0, w, h);
glMatrixMode(GL_PROJECTION);
glLoadIdentity();
glOrtho(-1.0, 1.0, -1.0, 1.0, -1.0, 1.0);
glMatrixMode(GL_MODELVIEW);

and

m_proj.setToIdentity();
m_proj.perspective(45.0f, GLfloat(w) / h, 0.01f, 100.0f);

makopo

I fixed the issue with calling QWidget::update() in QOpenGLWidget::paintGL(). Now the videostream works.

SGaist

Glad you found a solution and thanks for sharing !

Note that this is a rather unexpected solution has you should not need to do that.