Qt3d + Oculus SDK

njbrown · wrote on 22 Jun 2016, 08:40

@Dabulla you're right, Qt3D uses synchronous rendering when used in a QQuickView, but it did not work in the c++ version the last time I tried. The official version of Qt3D has come out since, so i'll have to try again and tell you guys if it works or not. It's probably the best option for getting vr working right now because you can definitely know when a frame starts and when it ends.

Another hackish option is to use a QFrameAction component to submit and swap the render textures each frame....but that'll only work if QFrameAction runs its callback on the same thread as the renderer. I'll have to investigate this further.

Also, subclassing the QRenderAspect wouldnt do much since the actual rendering takes place in another class called Renderer which isnt publicly accessible. So that option wouldnt work.

njbrown · wrote on 19 Aug 2016, 15:03

Has anyone found a creative solution to this problem?

jahshaka · 7 Oct 2016, 08:19

@Dabulla nick - njbrown - is working on a new 3d framework built on top of qt and qopengl that will have native support for the occulus so hit him up, it would be great if you can hep out! Will probably call it Qt3DC (for community edition since Qt3D seems to be enterprise only)

tekojo · wrote on 7 Oct 2016, 08:19

@jahshaka
Qt3D isn't enterprise only, it's all part of Qt.
KDAB just have been contributing most to it, and their priorities might not match yours (different kind of target applications bring up different kind of bugs).
So it would probably make more sense to align development efforts in one place and not build another module that for the most part duplicates work. Especially since you would know what to fix, getting patches in is not overly difficult.

kshegunov · 10 Jul 2016, 23:47

Actually @tekojo's wrong (wink), Qt3D desperately needs better docs for the features it has. So it's not even required you to learn its internals, just contributing know-how would greatly improve the module and your understanding of it. I myself have asked on several occasions the lead dev on the mailing how things are done and he's been responsive and understanding. I think you went the wrong way about it if you had expected him/them to give you code/help with your release - that'd require time and time costs are rather steep in our tidy little field. Instead I'd focus on seeking knowledge about the module - how things are done and putting examples/docs with that new knowledge would be very, very much welcome. I'm quite convinced Sean would jump at the occasion of someone helping with the docs (it's one of the reasons why Qt3D is opensource).

jahshaka · 13 Oct 2016, 13:29

@kshegunov we have tried to get help many times and received quotes for paid services on two occasions and zero help whatsoever - not even answers to basic questions. this is totally so un open source its not funny.

Sean has has not answered any of our posts here or on the mailing list thats for sure... and others have the same issue

we have already started on our engine (and it already has oculus support!!) and since its built on QOpenGL it should be easy for others to use and get involved... we hope a community built and supported engine will find a better home and better support

as soon as we have something worthy of publishing we will post a link to our github

kshegunov · wrote on 13 Oct 2016, 13:29

@jahshaka said in Qt3d + Oculus SDK:

Sean has has not answered any of our posts here or on the mailing list thats for sure... and others have the same issue

I'm far away from that particular project, much less I want to appear as trying to defend him/kdab. I just want the record to reflect that this is a user forum, devs don't frequent it. As for the mailing list, I see a single question (6th of May) posted, which in all fairness indeed did not get any attention.

Good luck with your engine! I will follow its development with great interest, although I haven't any work with 3D or graphics.
Kind regards.

jahshaka · wrote on 15 Oct 2016, 13:19

We are making good progress with our engine - and already have basic VR support

Dabulla · 6 Nov 2016, 20:49

I just got the first Qt3D Scene on the Oculus with headtracking! It required a small patch in qtbase (qopenglframebufferobject ). I think my patch would not survive a review yet. I use Scene3D with QQuickRendercontrol. It is a proof of concept and I it looks like it is locked at 60Hz at the moment.
I think a bit of engineering is needed to decouple everything from vsync.

? Offline · wrote on 6 Nov 2016, 20:49

@Dabulla Awesome!

Dabulla · 12 Aug 2016, 14:21

This is my code (proof of concept):

https://github.com/dabulla/qtbase/tree/virtual-reality
https://github.com/dabulla/qt3d/tree/virtual-reality

both are based on the v5.8.0-beta1 branches.

About:

you may need to change the path of the ovr lib in scene3d-vr.pro
example is located in qt3d/examples/qt3d/scene3d-vr and should work after qtbase was recompiled with my patch. I open example.pro in QtCreator.
The patch in qtbase allows to create an attachment for QOpenGLFramebufferObject by providing a GLuint.
The only classes used for VR at the moment are
- VrRenderer: basically Initialization and swapping buffer to HMD.
- VrCamera: Headtracking, stereo viewports
In WindowMultiThread::setHMDSurface(...) the FrameGraph is searched for a SurfaceSelector and a VrCamera. The SurfaceSelector is reconfigured from outside Qml to choose an offscreen surface. The VrCamera is read to initialize it with ovr and the current HMD. (These are workarounds because I do not yet know about adding aspects. I think there is a nicer solution by using the Entity Component System correctly).
The sample is a combination of qtdeclarative/examples/quick/rendercontrol and the qt3d scene3d example.
- Initialization for VR is done the first time "render" is called.
- Rendering happens into the offscreen texture that comes from ovr using the patched QOpenGLFramebufferObject.
- QuickRenderer additionally calls methods from ovr to "commit" textures to the headset (I think comparable to glSwapBuffers)

Known Issues:

I think rendering is vsynced to the monitor and window at the moment (60Hz).
Oculus only. I just had a closer look at OpenVR after a while and noticed, that it might be able to use a QOpenGLTexture directly! Thus, the patch in qtbase would not be needed for OpenVR. Although I think the Oculus API is a bit ahead of OpenVR at the moment and native support would be desirable.
seg fault sometimes on exit of the application. (maybe the offscreen texture is not cleaned up in the correct order)
No tracked controllers yet, no positional sound, no qml-interface for chaperone/guardian
Im not yet sure how to implement the patch in QOpenGLFramebufferObject cleanly. I want to avoid calling glGet* too often, only to make calls to format() possible. This means QOpenGLFramebufferObject is not constructed correctly. On the other hand side QOpenGLFramebufferObject must be used in order to call QQuickWindow::setRenderTarget(...).

Suggestions are very welcome.

My next step would be to get the example working on the HMDs correct refresh rate and then implement this in a clean way respecting Qt Coding Style. There should be a common vr interface (c++/qml) which can have OpenVR/OculusVR/OSVR as a backend. Then there should be a working demo for the Vive (I heard there is a Vive in one of the KDAB offices :) ).

jahshaka · wrote on 5 Jan 2017, 01:11

in our quest for proper VR support, and since we could not afford the high support fees KDAB has out there we have built our own Qt 3D VR engine and it works pretty well...

moved it into its own thread here so please jump in and help out !!!

https://forum.qt.io/topic/74807/irisgl-alpha-code-up