[SOLVED] Using a vtk renderer in QtQuick

z.emb

Is it possible to use a vtk renderer or render window in the qml scene graph?

I know there is a widget based solution for that, like QVTKWidget or the vtkGenericOpenGLRenderWindow, which can be used to render vtk objects in a QGLWidget. But is there a chance to use vtk with qml?
It would be really nice to use the vtk scene as a parent qml item of a scene (with all the heavy filter calculations) and put some qml UI controlls on top of that.

Help would be appreciated.

Zamx

To show Ogre3D as a qml item, I render it off screen to a texture and then show the texture. Maybe you can do the same for vtk renderer?

Here you can find the code for ogre : "qmlogre":http://qt.gitorious.org/qt-labs/qmlogre

z.emb

Thanks for that usefull information.

I will provide the info I found out so far ():
The vtkWindowToImageFilter class can be used to render a vtkRenderWindow to a vtkImageData, from which a OpenGL texture can be created somehow (I haven't figured out how this is done).

Zamx

In Qt 5 you can QQuickview to create a QSGTexture from a QImage. This QSGTexture can be added to a QSGGeometryNode and you can add the QSGGeometryNode to a QQuickItem.

z.emb

Thanks again for your help.

Thats definitively one thing I will try. But I will also try another approach:
As I wrote in the first post, the vtkGenericOpenGLRenderWindow is used to render vtk objects directly in a QGLWidget, so I think it should be possible to derive from QQuickView and make use of its QOpenGLContext property, which seems to provide a similar interface to the OpenGL context like the QGLWidget.

I will post the results, maybe this helps other people with a similar task.

z.emb

I ran into some problems using QQuickView.

I subclassed QQuickView to trigger the vtk rendering, but I have problems to get the OpenGL context.

For example, in a slot that is connected to the sceneGraphInitialized() signal, where I want to do my own initialization, glGetString(GL_VERSION) return always a NULL pointer.

Do I miss something? How can I use the gl*-functions in the subclassed QQuickView?

Zamx

To get the OpenGL context I use QGLContext *ctx = QGLContext::currentContext() from "QGLFunctions":http://qt-project.org/doc/qt-5.0/qglfunctions.html. I use it in a slot connected to the sceneGraphInitialized() signal.

z.emb

Thanks for your answer. But the problem was that the precompiled windows version of the Qt5 Beta2 has ANGLE enabled and vtk uses the native OpenGL API.

I compiled Qt5 with the '-opengl desktop' flag enabled and the problem is gone.

z.emb

Just a quick update.
Thanks to the nice guys in the qt-dev IRC I was able to successfully render a vtkRenderWindow in a subclass of QQuickItem/QSGNode using a FBO.

Hopefully I can publish the code in the near future, after I integrated the vtk interaction concept and cleaned up the code.

If someone interested I can PM the current state of the code.

Alexey

[quote author="z.emb" date="1360680945"]Just a quick update.
Thanks to the nice guys in the qt-dev IRC I was able to successfully render a vtkRenderWindow in a subclass of QQuickItem/QSGNode using a FBO.

If someone interested I can PM the current state of the code.[/quote]

Hi, could you, please, provide here only code snippets for the following steps:

creating FBO
rendering into it
create and initialize QSGNode using FBO

?

z.emb

I subclassed QSGSimpleTextureNode, that provides some basic functionality like geometry and material handling. Basically I had only to implement two methods. First the virtual preprocess() method in which the rendering happens (remark that the preprocess method runs in the rendering thread and the GUI thread is not blocked! You need also set the preprocess flag in the constructor).

@
void QVtkQuickRenderWindowNode::preprocess()
{
if(d_data->glFrameBufferObject == nullptr)
return;
//Incompatibility between GLSL and the OpenGL 1.x api used by vtk
//The qml scene graph seems to use a gl-programm and does not unload it!?
QOpenGLContext::currentContext()->functions()->glUseProgram(0);
glPopAttrib();
d_data->glFrameBufferObject->bind();

//Do your rendering and native openGL calls


glPushAttrib(GL_ALL_ATTRIB_BITS);
d_data->glFrameBufferObject->release();

}
@

and second my own updateNode() method where I create the FBO and the FBO texture object if necessary.

@
void QVtkQuickRenderWindowNode::updateNode()
{
//initialize first if necessary
if(!isInitialized())
initialize();

//check if geometry is marked dirty or fbo is empty
if(((dirtyState() & DirtyGeometry) != 0) || (d_data->glFrameBufferObject == nullptr)) {
    //create new framebuffer and texture (only if size is valid)
    d_data->glFrameBufferObject.reset(validSize() ? new QOpenGLFramebufferObject(d_data->size, QOpenGLFramebufferObject::Depth) : nullptr);
    d_data->fboTexture.reset(validSize() ? new QSGFboTexture(d_data->glFrameBufferObject.get()) : nullptr);
}

setTexture(d_data->fboTexture.get());

}
@

Subclassing the QSGTexture class (in my case QSGFboTexture) was pretty self explanatory. Just implement the pure virtual methods (calling glBindTexture(GL_TEXTURE_2D, fboTextureId) in the bind() method is important).

The updatePaintNode() method in the derived QQuickItem class just looks like this:
@
QSGNode* QVtkQuickRenderWindowItem::updatePaintNode(QSGNode* oldNode, UpdatePaintNodeData* updatePaintNodeData)
{
qDebug() << "UPDATE_NODE" << boundingRect();

if (width() <= 0 || height() <= 0) {
    delete oldNode;
    return nullptr;
}

QVtkQuickRenderWindowNode* renderWindowNode = static_cast<QVtkQuickRenderWindowNode*>(oldNode);
if (renderWindowNode == nullptr)
    renderWindowNode = new QVtkQuickRenderWindowNode();

renderWindowNode->setSize(boundingRect().size().toSize());
renderWindowNode->setRect(boundingRect());
renderWindowNode->updateNode();
    
return renderWindowNode;

}
@

I think there will be a example regarding rendering a FBO as texture in the next Qt minor release. The guy in the IRC opened a Bug ( https://bugreports.qt-project.org/browse/QTBUG-29548 ) after I asked all my stupid question in the IRC :-)

Alexey

[quote author="z.emb" date="1360745900"]I subclassed QSGSimpleTextureNode, that provides some basic functionality like geometry and material handling. Basically I had only to implement two methods. First the virtual preprocess() method in which the rendering happens (remark that the preprocess method runs in the rendering thread and the GUI thread is not blocked! You need also set the preprocess flag in the constructor).

@
void QVtkQuickRenderWindowNode::preprocess()
{
if(d_data->glFrameBufferObject == nullptr)
return;
//Incompatibility between GLSL and the OpenGL 1.x api used by vtk
//The qml scene graph seems to use a gl-programm and does not unload it!?
QOpenGLContext::currentContext()->functions()->glUseProgram(0);
glPopAttrib();
d_data->glFrameBufferObject->bind();

//Do your rendering and native openGL calls


glPushAttrib(GL_ALL_ATTRIB_BITS);
d_data->glFrameBufferObject->release();

}
@

and second my own updateNode() method where I create the FBO and the FBO texture object if necessary.

@
void QVtkQuickRenderWindowNode::updateNode()
{
//initialize first if necessary
if(!isInitialized())
initialize();

//check if geometry is marked dirty or fbo is empty
if(((dirtyState() & DirtyGeometry) != 0) || (d_data->glFrameBufferObject == nullptr)) {
    //create new framebuffer and texture (only if size is valid)
    d_data->glFrameBufferObject.reset(validSize() ? new QOpenGLFramebufferObject(d_data->size, QOpenGLFramebufferObject::Depth) : nullptr);
    d_data->fboTexture.reset(validSize() ? new QSGFboTexture(d_data->glFrameBufferObject.get()) : nullptr);
}

setTexture(d_data->fboTexture.get());

}
@

[/quote]

Thank you very much! What is d_data? I'm very new to OpenGL...

[quote author="z.emb" date="1360745900"]I think there will be a example regarding rendering a FBO as texture in the next Qt minor release. The guy in the IRC opened a Bug ( https://bugreports.qt-project.org/browse/QTBUG-29548 ) after I asked all my stupid question in the IRC :-)[/quote]

How I need them now! :-)

z.emb

That is just a private member variable where I store all the variables. See "here":http://en.wikipedia.org/wiki/Pimpl for more information. This has nothing to do with OpenGL.

You can also add a normal QOpenGLFramebufferObject* and QSGTexture* pointer to your class.

Martell Malone

Hi z.emb,

I'm working on this exact problem also.

Could you tell me what QSGFboTexture is? and also fbo.get() fbo.reset() ?

z.emb

[quote author="Martell Malone" date="1360770728"]
What about setting the material and the opaque material do you even use them?

Is there anything else we should know to get this working?
I seem to be missing something as it keeps crashing.[/quote]

No I don't use them. I derived my node from QSGSimpleTextureNode, which handles these things for me, I think. I just need to add my own QSGFboTexture and thats it.

Like I said above, be sure that you don't access any objects in the preprocess() that are also accessed from the GUI thread.
For debugging reasons you can do the rendering also in the updatePaintNode() method (in my example in the updateNode() method which is called from updatePaintNode() ) to be sure it is not a threading problem.

Edit:
[quote author="Martell Malone" date="1360770728"]
Could you tell me what QSGFboTexture is? and also fbo.get() fbo.reset() ?
[/quote]

QSGFboTexture is my subclass for the FBO texture, just implement the pure virtual functions of QSGTexture:

@
void QSGFboTexture::setFbo(QOpenGLFramebufferObject* fbo)
{
if(d_data->fbo == fbo)
return;

d_data->fbo = fbo;
d_data->firstTimeBound = true;

}

void QSGFboTexture::bind()
{
if(d_data->fbo == nullptr)
return;

auto fboTexture = textureId();
glBindTexture(GL_TEXTURE_2D, fboTexture);

if(d_data->firstTimeBound) {
    updateBindOptions(true);
    d_data->firstTimeBound = false;
}

}

bool QSGFboTexture::hasAlphaChannel() const
{
return true;
}

bool QSGFboTexture::hasMipmaps() const
{
if(d_data->fbo == nullptr)
return false;

return d_data->fbo->format().mipmap();

}

int QSGFboTexture::textureId() const
{
if(d_data->fbo == nullptr)
return 0;

return d_data->fbo->texture();

}

QSize QSGFboTexture::textureSize() const
{
if(d_data->fbo == nullptr)
return QSize();

return d_data->fbo->size();

@

I use the "std::unique_ptr":http://www.cplusplus.com/reference/memory/unique_ptr/ template. Therefore the get() and reset().

Martell Malone

Thansk for that z.emb

I was working basing my code from qt3d before which is why I couldn't get it to work
http://qt.gitorious.net/qt-labs/qt3d/blobs/83fd6411a16f09b81252d4a4398a1968451754c9/src/imports/threed/viewportfbonode_sg.cpp

Seems you found the solution ;)

Martell Malone

glPopAttrib and glPushAttrib are invalid functions?

You can do beginNativePainting within QPainter to access these functions but I'm guessing thats not what you did.

Im on qt 5.0.1 ANGLE GLES version

What header did you include?

Martell Malone

I've made more progress on getting it working on GLES systems.
My window flickers for a frame and then goes gray.

Also for that frame the scene is upside down and a texture is missing.

The last two things that I wish to ask

I noticed
setFbo isn't called anywhere?

and also I have two size methods setRect and setSize

@void ColorNode::setSize(const QSize size)
{
if (size == m_size) return;

m_size = size;
//m_fbo = NULL;
//m_texture = NULL;

QSGGeometry::updateTexturedRectGeometry(&m_geometry,
                                            QRectF(0, 0, m_size.width(), m_size.height()),
                                            QRectF(0, 0, 1, 1));

markDirty(DirtyGeometry);

}

void ColorNode::setRect(const QRectF rect)
{
if (m_rect == rect) return

    markDirty(DirtyGeometry);
    m_rect = rect;

}@

I'm guessing yous implementation is different?
Could I get a quick look :)

I think I'm really close

z.emb

Actually, I posted almost the whole implementation already, but here you go: "Node":http://pastebin.com/8CgheNzm . The implementation of the texture and the item are already posted.

bq. I noticed setFbo isn’t called anywhere?

You may have noticed that I pass the FBO to the constructor of the texture ;-)

bq. What header did you include?

I compiled Qt 5.0.0 with ANGLE deactivated and the -opengl desktop flag, because I use VTK which also uses the native OpenGL system.

Martell Malone

I need to set the geometry in my setsize or I get nothing in the window.

@
QSGFboTexture::QSGFboTexture(QGLFramebufferObject* fbo)
:QSGTexture()
{
m_fbo = fbo;

// setFbo(fbo);
if (m_fbo != NULL)
firstTimeBound = true;

}
@

same as setFbo ;-)

The problem is that on ANGLE we are using a GLES emulator not OpenGL.
Those two functions don't exsist.

What happens to your window when you dont call the push and pop?

@void ColorNode::preprocess()
{
if(m_fbo == NULL) return; //will happen a few times

QOpenGLContext::currentContext()->functions()->glUseProgram(0);
m_fbo->bind();

CCDirector::sharedDirector()->mainLoop();

m_fbo->release();

m_fbo->toImage().save("name.png","png");

}@