Invalid VkSurface from QWidget

Desperado17

I try to create a valid VkSurfaceKHR surface object in a Qt 5.10.1 C++ application on Ubuntu Linux 18.04 for a QWidget derived class. The code for obtaining the vulkan instance and surface looks like this:

QVulkanInstance inst;

inst.setLayers(QByteArrayList() << "VK_LAYER_KHRONOS_validation");

if (!inst.create())
   qFatal("Failed to create Vulkan instance: %d", inst.errorCode());


   …  A QtWidget is created. code is omitted …

widget->show();

QWindow* realWindow = widget->windowHandle();

if ( !realWindow && widget->nativeParentWidget() )
    realWindow = widget->nativeParentWidget()->windowHandle();

realWindow->setSurfaceType ( QSurface::VulkanSurface );
realWindow->setVulkanInstance ( &inst );

m_VulkanSurface = QVulkanInstance::surfaceForWindow ( realWindow );

After successfully creating a vulcan instance and creating a device from it, I get a segfault in a vkGetPhysicalDeviceSurfaceSupportKHR call with this surface albeit surfaceForWindow doesn't return 0.

Shortly before the crash I get these error messages from VK_LAYER_KHRONOS_validation_layer:

vkDebug: Validation: 0: Validation Error: [ VUID-vkGetPhysicalDeviceSurfaceSupportKHR-surface-parameter ] Object 0: VK_NULL_HANDLE, type = VK_OBJECT_TYPE_INSTANCE; | MessageID = 0x801f247e | Invalid VkSurfaceKHR Object 0x55555777b560. The Vulkan spec states: surface must be a valid VkSurfaceKHR handle (https://vulkan.lunarg.com/doc/view/1.2.154.0/linux/1.2-extensions/vkspec.html#VUID-vkGetPhysicalDeviceSurfaceSupportKHR-surface-parameter)

vkDebug: Validation: 0: Validation Error: [ UNASSIGNED-Threading-Info ] Object 0: handle = 0x55555777b560, type = VK_OBJECT_TYPE_SURFACE_KHR; | MessageID = 0x5d6b67e2 | Couldn't find VkSurfaceKHR Object 0x55555777b560. This should not happen and may indicate a bug in the application.
Is my method for generating a Vulkan surface on Qt applicable?

Desperado17

I now enabled symbols and used valgrind. It seems like the handle returned in QWindow::handle() is not null but still invalid. Is this a bug or my doing?

==30713== Invalid read of size 8
==30713== at 0x1210B3A6: QXcbVulkanWindow::surface() (qxcbvulkanwindow.cpp:74)
==30713== by 0x120DE73E: QXcbNativeInterface::nativeResourceForWindow(QByteArray const&, QWindow*) (qxcbnativeinterface.cpp:268)
==30713== by 0x9CE3688: QVulkanInstance::surfaceForWindow(QWindow*) (qvulkaninstance.cpp:848)
==30713== by 0x26163A: MainWindow::on_btnCreateView_clicked() (mainwindow.cpp:551)
==30713== by 0x2876E2: qt_static_metacall (moc_mainwindow.cpp:296)
==30713== by 0x2876E2: MainWindow::qt_metacall(QMetaObject::Call, int, void**) (moc_mainwindow.cpp:297)
==30713== by 0xA0E97C1: QMetaObject::metacall(QObject*, QMetaObject::Call, int, void**) (qmetaobject.cpp:301)
==30713== by 0xA11027F: QMetaObject::activate(QObject*, int, int, void**) (qobject.cpp:3782)
==30713== by 0xA110514: QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (qobject.cpp:3629)
==30713== by 0x93EC011: QAbstractButton::clicked(bool) (moc_qabstractbutton.cpp:308)
==30713== by 0x93EC259: QAbstractButtonPrivate::emitClicked() (qabstractbutton.cpp:414)
==30713== by 0x93ED8C3: QAbstractButtonPrivate::click() (qabstractbutton.cpp:407)
==30713== by 0x93EDB7D: QAbstractButton::mouseReleaseEvent(QMouseEvent*) (qabstractbutton.cpp:1011)
==30713== Address 0x29966960 is 0 bytes after a block of size 176 alloc'd
==30713== at 0x4C3217F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==30713== by 0x584FD98: QXcbGlxIntegration::createWindow(QWindow*) const (qxcbglxintegration.cpp:178)
==30713== by 0x120B918A: QXcbIntegration::createPlatformWindow(QWindow*) const (qxcbintegration.cpp:245)
==30713== by 0x999AE22: QWindowPrivate::create(bool, unsigned long long) (qwindow.cpp:509)
==30713== by 0x999AF44: QWindow::create() (qwindow.cpp:629)
==30713== by 0x9312914: QWidgetPrivate::create_sys(unsigned long long, bool, bool) (qwidget.cpp:1483)
==30713== by 0x9312ED9: QWidget::create(unsigned long long, bool, bool) (qwidget.cpp:1337)
==30713== by 0x931ECB2: QWidget::setVisible(bool) (qwidget.cpp:8280)
==30713== by 0x931BAD7: QWidget::show() (qwidget.cpp:7883)
==30713== by 0x2615CB: MainWindow::on_btnCreateView_clicked() (mainwindow.cpp:536)
==30713== by 0x2876E2: qt_static_metacall (moc_mainwindow.cpp:296)
==30713== by 0x2876E2: MainWindow::qt_metacall(QMetaObject::Call, int, void**) (moc_mainwindow.cpp:297)
==30713== by 0xA0E97C1: QMetaObject::metacall(QObject*, QMetaObject::Call, int, void**) (qmetaobject.cpp:301)
==30713==
==30713== Invalid write of size 8
==30713== at 0x1210B3FA: QXcbVulkanWindow::surface() (qxcbvulkanwindow.cpp:83)
==30713== by 0x120DE73E: QXcbNativeInterface::nativeResourceForWindow(QByteArray const&, QWindow*) (qxcbnativeinterface.cpp:268)
==30713== by 0x9CE3688: QVulkanInstance::surfaceForWindow(QWindow*) (qvulkaninstance.cpp:848)
==30713== by 0x26163A: MainWindow::on_btnCreateView_clicked() (mainwindow.cpp:551)
==30713== by 0x2876E2: qt_static_metacall (moc_mainwindow.cpp:296)
==30713== by 0x2876E2: MainWindow::qt_metacall(QMetaObject::Call, int, void**) (moc_mainwindow.cpp:297)
==30713== by 0xA0E97C1: QMetaObject::metacall(QObject*, QMetaObject::Call, int, void**) (qmetaobject.cpp:301)
==30713== by 0xA11027F: QMetaObject::activate(QObject*, int, int, void**) (qobject.cpp:3782)
==30713== by 0xA110514: QMetaObject::activate(QObject*, QMetaObject const*, int, void**) (qobject.cpp:3629)
==30713== by 0x93EC011: QAbstractButton::clicked(bool) (moc_qabstractbutton.cpp:308)
==30713== by 0x93EC259: QAbstractButtonPrivate::emitClicked() (qabstractbutton.cpp:414)
==30713== by 0x93ED8C3: QAbstractButtonPrivate::click() (qabstractbutton.cpp:407)
==30713== by 0x93EDB7D: QAbstractButton::mouseReleaseEvent(QMouseEvent*) (qabstractbutton.cpp:1011)
==30713== Address 0x29966960 is 0 bytes after a block of size 176 alloc'd
==30713== at 0x4C3217F: operator new(unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==30713== by 0x584FD98: QXcbGlxIntegration::createWindow(QWindow*) const (qxcbglxintegration.cpp:178)
==30713== by 0x120B918A: QXcbIntegration::createPlatformWindow(QWindow*) const (qxcbintegration.cpp:245)
==30713== by 0x999AE22: QWindowPrivate::create(bool, unsigned long long) (qwindow.cpp:509)
==30713== by 0x999AF44: QWindow::create() (qwindow.cpp:629)
==30713== by 0x9312914: QWidgetPrivate::create_sys(unsigned long long, bool, bool) (qwidget.cpp:1483)
==30713== by 0x9312ED9: QWidget::create(unsigned long long, bool, bool) (qwidget.cpp:1337)
==30713== by 0x931ECB2: QWidget::setVisible(bool) (qwidget.cpp:8280)
==30713== by 0x931BAD7: QWidget::show() (qwidget.cpp:7883)
==30713== by 0x2615CB: MainWindow::on_btnCreateView_clicked() (mainwindow.cpp:536)
==30713== by 0x2876E2: qt_static_metacall (moc_mainwindow.cpp:296)
==30713== by 0x2876E2: MainWindow::qt_metacall(QMetaObject::Call, int, void**) (moc_mainwindow.cpp:297)
==30713== by 0xA0E97C1: QMetaObject::metacall(QObject*, QMetaObject::Call, int, void**) (qmetaobject.cpp:301)

Desperado17

So we're getting closer:
It seems that my widget and all related native windows have the surface Type "RasterGLSurface". It seems that QWidgetWindow sets this by default in constructor. This prevents QXcbVulkanWindow::surface() from creating an actual Vulkan surface.
Is there a way to make a widget and its window recreate the surface?

kshegunov

Of course.
Create a Vulkan window[1] and embed it as a widget[2].

[1] https://doc.qt.io/qt-5/qvulkanwindow.html
[2] https://doc.qt.io/qt-5/qwidget.html#createWindowContainer

Desperado17

I avoided QVulkanWindow for now because it is too heavyweight implementing a lot of stuff I'd rather perform in the engine. I'm currently experimenting with this setup:

Create a small class that derives from QWindow that does nothing but set the surface type to VulkanSurface in the constructor.
Turn the window into a widget through QWidget::createWindowContainer
Replace former widget using ui->glControls->replaceWidget.

I was able to obtain a valid VkSurface from the QWindow dervative.

Desperado17

I still have difficulties getting the vulkan window surface to show up in the widget. The surface should be there, it occupies the area in the layout. But I still only get the gray qt widget background, nothing that indicates the presence of a framebuffer or the clear commands I issued to the queue.
What is the minimum amount of Vulkan commands that has to be issued in order to be sure it replaces the contents of the widget? Is there anything I have to do after the vkQueuePresentKHR to make sure my window sees it?

Desperado17

Ok, it seems to render if I create the vulkan enabled QWindow as a standalone window. But if I turn it into a widget using QWidget::createWindowContainer it won't.