Layering Qt Windows with buildroot DRM/KMS

[DMacAttack]

My project uses buildroot on a rockchip rk3288 processor (Tinkerboard). My project uses Qt QML with Qt version 5.9.1. I am using the libmali-<version>-gbm.so library, since provides me with eglfs_kms for my Qt app. I have used eglfs on an older framebuffer kernel on iMX6 devices, and its fast from bootup, simple, and doesnt depend on weston like wayland does.

I would like to layer two Qt QML windows, where the bottom layer is video, and the top layer is a GUI. Like this:

__________________________
|              ___________|_______________
|              |                          |
|  video       |                          |
|  layer       |                          |
|              |    ui  layer             |
|              |                          |
|______________|                          |
               |__________________________|

The UI layer will have a transparent background, so the video layer can be seen beneath. On my last project (iMX6) used an older kernel, and had framebuffers to do this.

QT_QPA_EGLFS_FB=/dev/fb0 QT_QPA_PLATFORM=eglfs ./bottomWindow & 
QT_QPA_EGLFS_FB=/dev/fb1 QT_QPA_PLATFORM=eglfs ./topWindow &

topWindow is basically an ApplicationWindow where the background is transparent:

// main.qml
ApplicationWindow 
{
    flags: Qt.FramelessWindowHint | Qt.QWindow
    visible: true
    width: 1280
    height: 720

    color: "transparent" 

    Text
    {
        anchors.centerIn: parent
        text: "Hello" 
    }
}

The reason I want separate Windows is because I need to take screenshots & videorecordings of the MyVideoApp that has transformations in the QML. I cannot do this if the video resides in a QML surface inside MyUIApp. On my iMX platform I simply scraped the framebuffer (/dev/fb0).

My new project does not use framebuffers, it uses the KMS/DRM platform. This is okay, but I am not an expert in KMS/DRM. Most of my knowledge comes from https://events.static.linuxfound.org/sites/events/files/slides/brezillon-drm-kms.pdf . I know that the DRM has a base framebuffer, and kms overlay planes. According to the Qt documentation https://doc.qt.io/qt-5/embedded-linux.html it is possible to allocate the qt window to a KMS plane, but the variables I am attempting are not working:

# export QT_LOGGING_RULES=qt.qpa.egl*=true
# ./bottomWindow &
qt.qpa.egldeviceintegration: EGL device integration plugin keys: ("eglfs_emu", "eglfs_kms_egldevice", "eglfs_kms", "eglfs_x11")
qt.qpa.egldeviceintegration: EGL device integration plugin keys (sorted): ("eglfs_kms", "eglfs_emu", "eglfs_kms_egldevice", "eglfs_x11")
qt.qpa.egldeviceintegration: Trying to load device EGL integration "eglfs_kms" 
qt.qpa.eglfs.kms: New DRM/KMS via GBM integration created
qt.qpa.egldeviceintegration: Using EGL device integration "eglfs_kms" 
qt.qpa.eglfs.kms: platformInit: Opening DRM device
qt.qpa.eglfs.kms: Found the following video devices: ("/dev/dri/card0")
qt.qpa.eglfs.kms: Using "/dev/dri/card0" 
qt.qpa.eglfs.kms: Using backend-provided DRM device /dev/dri/card0
qt.qpa.eglfs.kms: Creating GBM device for file descriptor 5 obtained from "/dev/dri/card0" 
qt.qpa.eglfs.kms: "DSI1" mode count: 1
qt.qpa.eglfs.kms: mode 0 720 x 1280 @ 61 hz
qt.qpa.eglfs.kms: Selected mode 0 : 720 x 1280 @ 61 hz for output "DSI1" 
qt.qpa.eglfs.kms: Physical size is QSizeF(0, 0) mm for output "DSI1" 
qt.qpa.eglfs.kms: Creating new global GBM mouse cursor
qt.qpa.eglfs.kms: Initializing cursor atlas from ":/cursor.json" 
qt.qpa.eglfs.kms: Sorted screen list: QVector(OrderedScreen(QPlatformScreen=0x177e568 ("DSI1") : 2147483647 / QPoint(0,0) / primary: false))
qt.qpa.eglfs.kms: Adding QPlatformScreen 0x177e568 ( "DSI1" ) to QPA with geometry QRect(0,0 720x1280) and isPrimary= false
qt.qpa.input: X-less xkbcommon not available, not performing key mapping
qt.qpa.eglfs.kms: Creating window for screen "DSI1" 
eglfs >> createSurface GBM_FORMAT_ARGB8888
eglfs >> sucess to create KMS FB!


# QT_QPA_EGLFS_LAYER_INDEX=1 ./topWindow 
qt.qpa.egldeviceintegration: EGL device integration plugin keys: ("eglfs_emu", "eglfs_kms_egldevice", "eglfs_kms", "eglfs_x11")
qt.qpa.egldeviceintegration: EGL device integration plugin keys (sorted): ("eglfs_kms", "eglfs_emu", "eglfs_kms_egldevice", "eglfs_x11")
qt.qpa.egldeviceintegration: Trying to load device EGL integration "eglfs_kms" 
qt.qpa.eglfs.kms: New DRM/KMS via GBM integration created
qt.qpa.egldeviceintegration: Using EGL device integration "eglfs_kms" 
qt.qpa.eglfs.kms: platformInit: Opening DRM device
qt.qpa.eglfs.kms: Found the following video devices: ("/dev/dri/card0")
qt.qpa.eglfs.kms: Using "/dev/dri/card0" 
qt.qpa.eglfs.kms: Using backend-provided DRM device /dev/dri/card0
qt.qpa.eglfs.kms: Creating GBM device for file descriptor 5 obtained from "/dev/dri/card0" 
qt.qpa.eglfs.kms: "DSI1" mode count: 1
qt.qpa.eglfs.kms: mode 0 720 x 1280 @ 61 hz
qt.qpa.eglfs.kms: Selected mode 0 : 720 x 1280 @ 61 hz for output "DSI1" 
qt.qpa.eglfs.kms: Physical size is QSizeF(0, 0) mm for output "DSI1" 
qt.qpa.eglfs.kms: Creating new global GBM mouse cursor
qt.qpa.eglfs.kms: Initializing cursor atlas from ":/cursor.json" 
qt.qpa.eglfs.kms: Sorted screen list: QVector(OrderedScreen(QPlatformScreen=0x14c1568 ("DSI1") : 2147483647 / QPoint(0,0) / primary: false))
qt.qpa.eglfs.kms: Adding QPlatformScreen 0x14c1568 ( "DSI1" ) to QPA with geometry QRect(0,0 720x1280) and isPrimary= false
qt.qpa.input: X-less xkbcommon not available, not performing key mapping
qt.qpa.eglfs.kms: Creating window for screen "DSI1" 
eglfs >> createSurface GBM_FORMAT_ARGB8888
eglfs >> sucess to create KMS FB!

As you can see the topWindow creates a new framebuffer and requests the DRM to switch to that new framebuffer. It does not use the KMS plane. I know for a fact that I have 4 kms planes :

GST_DEBUG="*:4" gst-launch videotestsrc ! kmssink 

Setting pipeline to PAUSED ...

INFO   GST_STATES gstbin.c:2506:gst_bin_element_set_state:<kmssink0> current NULL pending VOID_PENDING, desired next READY
INFO      kmssink gstkmssink.c:367:log_drm_version:<kmssink0> DRM v1.0.0 [rockchip — RockChip Soc DRM — 20140818]
INFO      kmssink gstkmssink.c:414:get_drm_caps:<kmssink0> prime import (✓) / prime export (✓) / async page flip (✓)
INFO      kmssink gstkmssink.c:569:ensure_allowed_caps:<kmssink0> ignoring format RG24
INFO      kmssink gstkmssink.c:569:ensure_allowed_caps:<kmssink0> ignoring format BG24
INFO      kmssink gstkmssink.c:569:ensure_allowed_caps:<kmssink0> ignoring format RG16
INFO      kmssink gstkmssink.c:569:ensure_allowed_caps:<kmssink0> ignoring format BG16
INFO      kmssink gstkmssink.c:569:ensure_allowed_caps:<kmssink0> ignoring format NV24
INFO      kmssink gstkmssink.c:569:ensure_allowed_caps:<kmssink0> ignoring format NA16
INFO      kmssink gstkmssink.c:569:ensure_allowed_caps:<kmssink0> ignoring format NA24
INFO      kmssink gstkmssink.c:686:gst_kms_sink_start:<kmssink0> connector id = 84 / crtc id = 74 / plane id = 75 <---------------plane id 75 being used
INFO      kmssink gstkmssink.c:708:gst_kms_sink_start:<kmssink0> display size: pixels = 720x1280 / millimeters = 0x0
INFO   GST_STATES gstelement.c:2676:gst_element_continue_state:<kmssink0> completed state change to READY

I have also tried setting both QT_QPA_EGLFS_KMS_CONNECTOR_INDEX=0 and QT_QPA_EGLFS_KMS_PLANE_INDEX=1 With the same result.

How do I get my qt app to launched on the kms plane ? Am I missing something in the Qt so that it doesn't create a new framebuffer?