Solved Achieving stable 60 fps framerate, QML combined with custom QQuickFrameBufferObject OpenGL Renderer

[ind1g1ne0us]

Hello, I'm developing an interactive, realtime running OpenGL application that needs to be rendered 60 fps. The basic structure I'm using is based on this project: https://github.com/KDAB/integrating-qq2-with-opengl

Which means, I'm using QT Quick 2 combined with a custom QQuickFramebufferObject class combined with a QQuickFramebufferObject::Renderer doing the OpenGL rendering.

This is what my main QML currently looks like:

import QtQuick 2.9
import QtQuick.Controls 2.2
import QtQuick.Layouts 1.3

import OmniGeometry 1.0
import "."

Rectangle {
    id: main_window
    visible: true
    focus: true

    width: 1243
    height: 700

    // This renders the whole scene.
    // Contains also the object properties that we are drawing
    // In reality though we should probably have some kind of object property separate from this ?
    // As we want to set properties intelligently ?
    // Or, we could have a Controller class that handles all the setting of the parameters
    // Let's just implement like a
    Controller {
        id: og_controller
        x: 0
        y: 0
        width: parent.width
        height: parent.height

        // For testing purposes this is now the model matrix
        // For the object we are rendering
        transform: [
            Rotation {
                origin.x: 0
                origin.y: 0
                axis { x: 0; y: 0; z: 1 }
                angle: 0
            },

            Scale {
                id: og_controller_scale
                origin.x: 0
                origin.y: 0
                xScale: 1.0
                yScale: 1.0
            },

            Translate {
                x: 0.0
                y: 0.0
            }
        ]
    }
}

So, it's very simple now, I'm still trying to figure out how to get stable 60 FPS rendering for this Controller class. I am registering the Controller in my main.cpp and also setting up the OpenGL context like this:

int main(int argc, char *argv[]) {
	QCoreApplication::setAttribute(Qt::AA_EnableHighDpiScaling);
	QGuiApplication app(argc, argv);

	QSurfaceFormat format;
	format.setDepthBufferSize(24);
	format.setStencilBufferSize(8);
	format.setSamples(8);

	// Request OpenGL 3.3 core or OpenGL ES 3.1
	if (QOpenGLContext::openGLModuleType() == QOpenGLContext::LibGL) {
		qDebug("Requesting 3.3 core context");
		format.setVersion(3, 3);
		format.setProfile(QSurfaceFormat::CoreProfile);
	} else {
		qDebug("Requesting 3.2 context");
		format.setVersion(3, 2);
	}

	qmlRegisterType<Controller>("OmniGeometry", 1, 0, "Controller");

	QQuickView view;
	view.setFormat(format);
	view.setSource(QUrl("qrc:///main.qml"));
	view.show();

	// Set so that cannot be resized
	QObject *root = view.rootObject();
	QSize size = {root->property("width").toInt(), root->property("height").toInt()};
	view.setMinimumSize(size);
	view.setMaximumSize(size);

	psilog(PSILog::MSG, "Application initialized!");

	return app.exec();
}

Then in my C++ code, I have the Controller class which is the QQuickFramebufferObject, and also handles the creation of all render objects and synchronizing the state to the QQuickFramebufferObject::Renderer class that is also created from the Controller.

I need to render stuff all the time at 60 FPS, as I have animations and other stuff that require the update() method to be called. Basically it's a realtime OpenGL scene, not based on just user input but having animated elements, like in a game, running all the time, and doing logic.

Currently I just create a QTimer in the Controller constructor method like this:

	_main_timer = new QTimer();
	if (_main_timer != nullptr) {
		_main_timer->setInterval((1.0f / 60.0f) * 1000.0f);

		connect(_main_timer, &QTimer::timeout,
			this, &Controller::update);

		_main_timer->start();
	}

This then calls the update() method which does this:

void Controller::update() {
	QQuickItem::update();

	static float last_elapsed_time;
	last_elapsed_time = _ctx.elapsed_time;
	_ctx.elapsed_time = (GLfloat)_elapsed_timer->elapsed();

	qDebug() << "elapsed_time = " << _ctx.elapsed_time << " time_diff " << (_ctx.elapsed_time - last_elapsed_time);
}

So basically how I understand this runs the event loop for the Controller. I have some code there to measure the elapsed times between calls, and they range wildly, sometimes the code runs at perfect 60 fps (mainly after starting up for a while) but quickly the rendering can get out of sync, and falls out of perfect smoothness.

Here's an example of the times when I run for some time:

elapsed_time =  9944  time_diff  15
elapsed_time =  9961  time_diff  17
elapsed_time =  9978  time_diff  17
elapsed_time =  9994  time_diff  16
elapsed_time =  10011  time_diff  17
elapsed_time =  10027  time_diff  16
elapsed_time =  10044  time_diff  17
elapsed_time =  10056  time_diff  12
elapsed_time =  10077  time_diff  21
elapsed_time =  10090  time_diff  13
elapsed_time =  10104  time_diff  14
elapsed_time =  10121  time_diff  17
elapsed_time =  10137  time_diff  16
elapsed_time =  10153  time_diff  16
elapsed_time =  10170  time_diff  17
elapsed_time =  10185  time_diff  15
elapsed_time =  10201  time_diff  16
elapsed_time =  10217  time_diff  16
elapsed_time =  10237  time_diff  20
elapsed_time =  10250  time_diff  13
elapsed_time =  10270  time_diff  20
elapsed_time =  10280  time_diff  10
elapsed_time =  10304  time_diff  24
elapsed_time =  10314  time_diff  10
elapsed_time =  10337  time_diff  23
elapsed_time =  10345  time_diff  8
elapsed_time =  10371  time_diff  26
elapsed_time =  10379  time_diff  8
elapsed_time =  10403  time_diff  24
elapsed_time =  10412  time_diff  9
elapsed_time =  10438  time_diff  26
elapsed_time =  10440  time_diff  2
elapsed_time =  10457  time_diff  17
elapsed_time =  10473  time_diff  16
elapsed_time =  10488  time_diff  15
elapsed_time =  10505  time_diff  17
elapsed_time =  10521  time_diff  16
elapsed_time =  10538  time_diff  17
elapsed_time =  10554  time_diff  16
elapsed_time =  10571  time_diff  17
elapsed_time =  10585  time_diff  14
elapsed_time =  10604  time_diff  19
elapsed_time =  10617  time_diff  13
elapsed_time =  10638  time_diff  21
elapsed_time =  10649  time_diff  11
elapsed_time =  10671  time_diff  22
elapsed_time =  10681  time_diff  10
elapsed_time =  10705  time_diff  24
elapsed_time =  10712  time_diff  7
elapsed_time =  10738  time_diff  26
elapsed_time =  10746  time_diff  8
elapsed_time =  10772  time_diff  26
elapsed_time =  10780  time_diff  8
elapsed_time =  10805  time_diff  25
elapsed_time =  10813  time_diff  8
elapsed_time =  10838  time_diff  25
elapsed_time =  10841  time_diff  3

So what happens here I guess is that the frame updating is going out of sync, although the timer is firing at constantly 16.667 milliseconds, at 60 fps. So I'm guessing this is causing the frame lag.

My Renderer is basically just synchronizing the state between the Controller and doing custom OpenGL, very simple scene for now still, so that shouldn't be the problem.

Any idea how I could ensure that I call update() so that it is synchronized to the vsync, or make it perfectly stable ? I've also tried to put the timer in the main.qml and set the callback there for Controller::update(), but it makes no difference.

[ind1g1ne0us]

I tried also to remove the constant timer updates, instead of calling my Controller::update() from the render objects when the animated properties are set, and still the framerate is not smooth. So, is my main object event loop being called out of sync, or any idea what is causing this ?

Running on macOS 10.13.2.

[ind1g1ne0us]

Also here is my Renderer main render method:

void OgRenderer::render() {
	//qDebug() << "render()";

	renderCtx ctx = get_render_ctx();

	STACK_PUSH(ctx.projection);
	ctx.projection.top().translate(0.0f, 0.0f, -1.0f);

	begin_render(ctx);
		render_layer(_layers.at(LAYER_BG), ctx);
		render_layer(_layers.at(LAYER_UI), ctx);
		render_layer(_layers.at(LAYER_HELPERS), ctx);
	end_render(ctx);

	STACK_POP(ctx.projection);

	ctx.elapsed_frames++;
	_window->resetOpenGLState();
}

So it's very basic now, begin_render() just setups the OpenGL rendering (clear, setup blending if needed etc) and end_render() is the opposite to that. I'm rendering with a modern OpenGL stack, so the rendering shouldn't be the problem, unless I need to call something to make sure the rendering is updated.

[ind1g1ne0us]

Okay, seems if I add an NumberAnimation to my main qml that is running and affecting a visual element, the QQuickFrameBufferobject is also being rendered correctly. So I'm suspecting the main QT SceneGraph is not updating the event loop if there is no animation running. Is there a way to force updating the scene graph or event loop ?

I'm planning to add QML objects on top of the OpenGL scene to control the rendering, so maybe I don't want to do that. What I would like to do is run 60 fps, vsync synchronized updating just for the Controller class instance.

EDIT: Maybe I have to re-think my approach, maybe I could do it more event based and call update() for each element that needs to be updated in my scene. Would still like to understand how to "force" the main event loop to be updated without having to control some dummy visual element.

[ind1g1ne0us]

Okay, a kind soul on the #qt channel on freenode helped me, the solution was to also call the window update() method along with QQuickItem::update(). So basically just calling _window->update() (window I stored from windowChanged() signal) in my Controller::update() method did the trick.

I setup again a timer running at 60 fps and calling the Controller::update() method, this does the trick. According to the documentation ""Calling QQuickWindow::update() differs from QQuickItem::update() in that it always triggers a repaint, regardless of changes in the underlying scene graph or not.""