vsync: missing rendered frames

geissenpeter

Hello,
for a scientific task, flickering areas with a stable frequency (max. 60 Hz), shall be displayed on the screen. I tried to achieve a stable stimulus visualization using Qt 5.6.

According to this blog entry and many other online recommendations, I realized three different approaches: Inheriting from QWindow Class, QOpenGLWindow Class and QRasterWindow Class. I wanted to get the accurate advantage of vsync and avoid the usage of QTimer.

The flickering area can be displayed. Also a stable time period between the frames has been measured with 16 up to 17 ms. But every few seconds some missed frames are spotted. It can be seen very clearly that there is no stable visualization of the stimulus. The same effect occurs on all three approaches.

Have I done the implementation of my code properly or do better solutions exist? If the code is adequate for its purpose do I have to assume that it is a hardware problem? Could it be that difficult then, to display a simple flickering area?

Thank you very much for helping me!

As Example you can see my code for QWindow Class here:

Window::Window(QWindow *parent)
: m_context(0)
, m_paintDevice(0)
, m_bFlickerState(true){
setSurfaceType(QSurface::OpenGLSurface);

QSurfaceFormat format;
format.setDepthBufferSize(24);
format.setStencilBufferSize(8);
format.setSwapInterval(1);
this->setFormat(format);

m_context.setFormat(format);
m_context.create();
}

void Window::render(){

//calculating elapsed time between frames
m_t1 = QTime::currentTime();
int curDelta = m_t0.msecsTo(m_t1);
m_t0 = m_t1;
qDebug()<< curDelta;

m_context.makeCurrent(this);

if (!m_paintDevice)
    m_paintDevice = new QOpenGLPaintDevice;
if (m_paintDevice->size() != size())
    m_paintDevice->setSize(size());

QPainter p(m_paintDevice);
// draw using QPainter
if(m_bFlickerState){
    p.setBrush(Qt::white);
    p.drawRect(0,0,this->width(),this->height());
}
p.end();
m_bFlickerState = !m_bFlickerState;
m_context.swapBuffers(this);

// animate continuously: schedule an update
QCoreApplication::postEvent( this, new QEvent(QEvent::UpdateRequest));
}

kshegunov

Hello,

this->setFormat(format);

this is the same as:

setFormat(format)

the context (this) is resolved automatically. This is not an error by any means, only a note.

if (!m_paintDevice)
    m_paintDevice = new QOpenGLPaintDevice;

this part I don't understand. Your QWindow (if Window is derived fromQWindow) is already a paint device, so why do you need another? Could you provide the class declaration as well? This also goes to Window::render, why do that. I'd derive from QOpenGLWindow and do the painting in a QOpenGLWindow::paintGL and implement the other needed protected methods as well - initializing the GL context and resize at least.

QCoreApplication::postEvent( this, new QEvent(QEvent::UpdateRequest));

This is the same as calling requestUpdate().

Kind regards.

geissenpeter

@kshegunov
(I looked it up but I couldn't see the QWindow is a paint device but QOpenGLWindow does inherit from QPaintDeviceWindow class!)

Thank you very much for the fast response!

If you would choose QOpenGLWIndow, I provide you with the declaration and Implementation of my QOpenGLWindow approach. Like I said, I have the same problem here: frames are clearly missing!
Also I can't display this class in Fullscreen-mode on another screen: This error message then is generated:

QPainter::begin(): QOpenGLPaintDevice's context needs to be current
QPainter::begin(): Returned false
QPainter::end: Painter not active, aborted

Here is now my full code:
mainwindow.cpp:

#include "mainwindow.h"
#include "QPainter"

#include <QOpenGLWindow>


MainWindow::MainWindow(QOpenGLWindow *parent)
    : m_bFlickerState(true)
{
    QSurfaceFormat format;
    format.setDepthBufferSize(24);
    format.setStencilBufferSize(8);
    format.setSwapInterval(1);
    setFormat(format);
    //continous update when frame was displayed
    connect(this, SIGNAL(frameSwapped()), this, SLOT(update()));
}

void MainWindow::resizeGL(int w, int h) {

}

void MainWindow::paintGL() {
    //calculating FPS
    m_t1 = QTime::currentTime();
    int curDelta = m_t0.msecsTo(m_t1);
    m_t0 = m_t1;
    qDebug()<< curDelta;
//    QOpenGLFunctions *f = context()->functions();
//    f->glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
// issue some native OpenGL commands
    QPainter p(this);
    // draw using QPainter
    if(m_bFlickerState){
            //f->glClearColor(1,1,1,1);
        p.fillRect(0,0,100,100,Qt::white);
    }
    else{
        //f->glClearColor(0,0,0,0);
        p.fillRect(0,0,100,100,Qt::black);

    }
    p.end();
    m_bFlickerState = !m_bFlickerState;
}

mainwindow.h:

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QtCore>
#include <QOpenGLPaintDevice>
#include <QOpenGLFunctions>
#include <QOpenGLWindow>

class MainWindow : public QOpenGLWindow, protected QOpenGLFunctions
{
    Q_OBJECT

public:
    explicit MainWindow(QOpenGLWindow *parent = 0 );
    ~MainWindow();

protected:
    void resizeGL(int w, int h);
    void paintGL();

private:
    QTime   m_t0;
    QTime   m_t1;
    bool m_bFlickerState;
    QOpenGLPaintDevice *m_paintDevice;
    QTimer  timer;
};

#endif // MAINWINDOW_H

main.cpp:

#include "mainwindow.h"
#include <QApplication>
#include <QScreen>
#include <QVBoxLayout>
#include <QLayout>

int main(int argc, char *argv[])
{
    QApplication a(argc, argv);

    QScreen *screen = QGuiApplication::screens()[1];

    //setup the window
    MainWindow w;
    w.setScreen(screen);
    w.showFullScreen();

    return a.exec();
}

kshegunov

@geissenpeter said:

I looked it up but I couldn't see the QWindow is a paint device but QOpenGLWindow does inherit from QPaintDeviceWindow class!

Yes indeed, that's a mistake on my part.

I provide you with the declaration and Implementation of my QOpenGLWindow approach

It looks good to me.

Like I said, I have the same problem here: frames are clearly missing!

The only thing I could think of is that the paint events might be getting compressed and some of the updates are skipped. Perhaps you could attach an additional receiver for the frameSwapped signal and see if that's the case - you'd expect to see update request and then a repaint and then another update request etc. If that's not the case, then this might very well be the reason.

Kind regards.

geissenpeter

@kshegunov said:

some of the updates are skipped

I think this can't be, because the time measured between the update calls is always about 16 to 18 ms. Could a problem with the frame buffer occur instead?

Also I have a problem with the QPainter: When my class is displayed in Fullscreen mode (like shown in the code above) then this error is generated. I have looked it up, but haven't found a solution yet

QPainter::begin(): QOpenGLPaintDevice's context needs to be current
QPainter::begin(): Returned false
QPainter::end: Painter not active, aborted

thank you for your fast response!

kshegunov

@geissenpeter

I think this can't be, because the time measured between the update calls is always about 16 to 18 ms.

This is of no consequence. Look here: http://doc.qt.io/qt-5/eventsandfilters.html#sending-events
update() posts an event on the queue for later processing and this event is subject to being compressed on occasion. I don't know if you can force the repaint however.

Could a problem with the frame buffer occur instead?

I'm by no means an expert, but it seems doubtful.

Also I have a problem with the QPainter: When my class is displayed in Fullscreen mode (like shown in the code above) then this error is generated. I have looked it up, but haven't found a solution yet

Possibly you're getting a paint event when the windows is not exposed. But I'm just guessing here.

geissenpeter

@kshegunov
I realized your approach: Every frameSwapped() signal is following an update request. So this code seems to do its job. I don't know what possible errors I could look for anymore...
Could other solutions exist for this (in my opinion) quite simple task?

kshegunov

@geissenpeter

I don't know what possible errors I could look for anymore...

Sadly, I don't know either.

Could other solutions exist for this (in my opinion) quite simple task?

I'm sure there are other solutions, but I don't know of any better way. Perhaps one of our more GL-versed members, like @Chris-Kawa, might take a look and suggest something.

Chris Kawa

V-sync is hard ;) Basically it's fighting with the inherent noisiness of the system. If the output shows 16-17 ms then that's the problem. 17 ms is too much. That's the skipping you see.

Couple of things to reduce that noise:

Don't do I/O in the render loop! qDebug()is I/O and it can block on all kinds of buffering shenanigans.
Testing V-sync under a debugger is useless. Debugging introduces all kinds of noise into your app. You should be testing v-sync in Release mode without debugger attached.
try not to use signals/slots/events if you can help it. They can be noisy i.e. call update() manually at the end of paintGL. You skip some overhead this way (not much but every bit counts).
If all you need is a flickering screen avoid QPainter. It's not exactly slow, but drop into the begin() method of it and see how much it actually does. OpenGL has fast, dedicated facilities to fill the buffer with a color. You might as well use it.

Not directly related, but it will make your code cleaner:

Use QElapsedTimer instead of manually calculating time intervals. Why re-invent the wheel.

Applying these bits I was able to remove the skipping from your example. Note that the skipping will occur in some circumstances, e.g. when you move/resize the window or when OS/other apps are busy doing something . You have no control over that.

Here are the modified code bits:

//mainwindow.h
    ...
    QElapsedTimer  m_timer;
};

//mainwindow.cpp
MainWindow::MainWindow(QOpenGLWindow *parent) : m_bFlickerState(true)
{
    QSurfaceFormat format;
    format.setDepthBufferSize(24);
    format.setStencilBufferSize(8);
    format.setSwapInterval(1);
    setFormat(format);
    m_timer.start(); //note that it doesn't really start any timers, just records current time
}

void MainWindow::paintGL()
{
    int ms_elapsed = timer.restart(); //you can store it somewhere to analyze later 

    auto f = context()->functions();
    if(m_bFlickerState)
        f->glClearColor(1.0f, 1.0f, 1.0f, 1.0f);
    else
        f->glClearColor(0.0f, 0.0f, 0.0f, 1.0f);

    f->glClear(GL_COLOR_BUFFER_BIT);

    m_bFlickerState = !m_bFlickerState;
    update(); //schedules next update directly, without going through signal dispatching
}

kshegunov

@Chris-Kawa
Chris, would running the rendering in a dedicated thread without event loop that forces rendering on a regular interval (for example a QThread subclass) be beneficial in this case? Or are the waiting routines (i.e. QThread::msleep/QThread::usleep/QSemaphore::tryAcquire) not precise enough for such things?

Again I'm just curious, so don't answer if you don't have the time. :)

Chris Kawa

@kshegunov
Any kind of sleeping/timer is no good for frame-perfect rendering. It doesn't matter in which thread you do it. There's just no solid way to synchronize these with the refresh rate.
You can't for example calculate "oh I rendered in 7ms so I have ~9.66ms left till the v-sync so I'll sleep for 9ms". That will miss most of the time. If any of these primitives is late even a microsecond for the v-sync it's all lost.
The only way to have any control over it is start drawing as soon as the v-sync is done and wait for the next one. Basically QOpenGLContext::swapBuffers is your only tool for that (done implicitly when you exit paintGL).

kshegunov

@Chris-Kawa
Fair enough. Thanks for taking the time.

Cheers!