[SOLVED] Using a vtk renderer in QtQuick



  • Is it possible to use a vtk renderer or render window in the qml scene graph?

    I know there is a widget based solution for that, like QVTKWidget or the vtkGenericOpenGLRenderWindow, which can be used to render vtk objects in a QGLWidget. But is there a chance to use vtk with qml?
    It would be really nice to use the vtk scene as a parent qml item of a scene (with all the heavy filter calculations) and put some qml UI controlls on top of that.

    Help would be appreciated.



  • To show Ogre3D as a qml item, I render it off screen to a texture and then show the texture. Maybe you can do the same for vtk renderer?

    Here you can find the code for ogre : "qmlogre":http://qt.gitorious.org/qt-labs/qmlogre



  • Thanks for that usefull information.

    I will provide the info I found out so far ():
    The vtkWindowToImageFilter class can be used to render a vtkRenderWindow to a vtkImageData, from which a OpenGL texture can be created somehow (I haven't figured out how this is done).



  • In Qt 5 you can QQuickview to create a QSGTexture from a QImage. This QSGTexture can be added to a QSGGeometryNode and you can add the QSGGeometryNode to a QQuickItem.



  • Thanks again for your help.

    Thats definitively one thing I will try. But I will also try another approach:
    As I wrote in the first post, the vtkGenericOpenGLRenderWindow is used to render vtk objects directly in a QGLWidget, so I think it should be possible to derive from QQuickView and make use of its QOpenGLContext property, which seems to provide a similar interface to the OpenGL context like the QGLWidget.

    I will post the results, maybe this helps other people with a similar task.



  • I ran into some problems using QQuickView.

    I subclassed QQuickView to trigger the vtk rendering, but I have problems to get the OpenGL context.

    For example, in a slot that is connected to the sceneGraphInitialized() signal, where I want to do my own initialization, glGetString(GL_VERSION) return always a NULL pointer.

    Do I miss something? How can I use the gl*-functions in the subclassed QQuickView?



  • To get the OpenGL context I use QGLContext *ctx = QGLContext::currentContext() from "QGLFunctions":http://qt-project.org/doc/qt-5.0/qglfunctions.html. I use it in a slot connected to the sceneGraphInitialized() signal.



  • Thanks for your answer. But the problem was that the precompiled windows version of the Qt5 Beta2 has ANGLE enabled and vtk uses the native OpenGL API.

    I compiled Qt5 with the '-opengl desktop' flag enabled and the problem is gone.



  • Just a quick update.
    Thanks to the nice guys in the qt-dev IRC I was able to successfully render a vtkRenderWindow in a subclass of QQuickItem/QSGNode using a FBO.

    Hopefully I can publish the code in the near future, after I integrated the vtk interaction concept and cleaned up the code.

    If someone interested I can PM the current state of the code.



  • [quote author="z.emb" date="1360680945"]Just a quick update.
    Thanks to the nice guys in the qt-dev IRC I was able to successfully render a vtkRenderWindow in a subclass of QQuickItem/QSGNode using a FBO.

    If someone interested I can PM the current state of the code.[/quote]

    Hi, could you, please, provide here only code snippets for the following steps:

    1. creating FBO
    2. rendering into it
    3. create and initialize QSGNode using FBO

    ?



  • I subclassed QSGSimpleTextureNode, that provides some basic functionality like geometry and material handling. Basically I had only to implement two methods. First the virtual preprocess() method in which the rendering happens (remark that the preprocess method runs in the rendering thread and the GUI thread is not blocked! You need also set the preprocess flag in the constructor).

    @
    void QVtkQuickRenderWindowNode::preprocess()
    {
    if(d_data->glFrameBufferObject == nullptr)
    return;
    //Incompatibility between GLSL and the OpenGL 1.x api used by vtk
    //The qml scene graph seems to use a gl-programm and does not unload it!?
    QOpenGLContext::currentContext()->functions()->glUseProgram(0);
    glPopAttrib();
    d_data->glFrameBufferObject->bind();

    //Do your rendering and native openGL calls
    
    
    glPushAttrib(GL_ALL_ATTRIB_BITS);
    d_data->glFrameBufferObject->release();
    

    }
    @

    and second my own updateNode() method where I create the FBO and the FBO texture object if necessary.

    @
    void QVtkQuickRenderWindowNode::updateNode()
    {
    //initialize first if necessary
    if(!isInitialized())
    initialize();

    //check if geometry is marked dirty or fbo is empty
    if(((dirtyState() & DirtyGeometry) != 0) || (d_data->glFrameBufferObject == nullptr)) {
        //create new framebuffer and texture (only if size is valid)
        d_data->glFrameBufferObject.reset(validSize() ? new QOpenGLFramebufferObject(d_data->size, QOpenGLFramebufferObject::Depth) : nullptr);
        d_data->fboTexture.reset(validSize() ? new QSGFboTexture(d_data->glFrameBufferObject.get()) : nullptr);
    }
    
    setTexture(d_data->fboTexture.get());
    

    }
    @

    Subclassing the QSGTexture class (in my case QSGFboTexture) was pretty self explanatory. Just implement the pure virtual methods (calling glBindTexture(GL_TEXTURE_2D, fboTextureId) in the bind() method is important).

    The updatePaintNode() method in the derived QQuickItem class just looks like this:
    @
    QSGNode* QVtkQuickRenderWindowItem::updatePaintNode(QSGNode* oldNode, UpdatePaintNodeData* updatePaintNodeData)
    {
    qDebug() << "UPDATE_NODE" << boundingRect();

    if (width() <= 0 || height() <= 0) {
        delete oldNode;
        return nullptr;
    }
    
    QVtkQuickRenderWindowNode* renderWindowNode = static_cast<QVtkQuickRenderWindowNode*>(oldNode);
    if (renderWindowNode == nullptr)
        renderWindowNode = new QVtkQuickRenderWindowNode();
    
    renderWindowNode->setSize(boundingRect().size().toSize());
    renderWindowNode->setRect(boundingRect());
    renderWindowNode->updateNode();
        
    return renderWindowNode;
    

    }
    @

    I think there will be a example regarding rendering a FBO as texture in the next Qt minor release. The guy in the IRC opened a Bug ( https://bugreports.qt-project.org/browse/QTBUG-29548 ) after I asked all my stupid question in the IRC :-)



  • [quote author="z.emb" date="1360745900"]I subclassed QSGSimpleTextureNode, that provides some basic functionality like geometry and material handling. Basically I had only to implement two methods. First the virtual preprocess() method in which the rendering happens (remark that the preprocess method runs in the rendering thread and the GUI thread is not blocked! You need also set the preprocess flag in the constructor).

    @
    void QVtkQuickRenderWindowNode::preprocess()
    {
    if(d_data->glFrameBufferObject == nullptr)
    return;
    //Incompatibility between GLSL and the OpenGL 1.x api used by vtk
    //The qml scene graph seems to use a gl-programm and does not unload it!?
    QOpenGLContext::currentContext()->functions()->glUseProgram(0);
    glPopAttrib();
    d_data->glFrameBufferObject->bind();

    //Do your rendering and native openGL calls
    
    
    glPushAttrib(GL_ALL_ATTRIB_BITS);
    d_data->glFrameBufferObject->release();
    

    }
    @

    and second my own updateNode() method where I create the FBO and the FBO texture object if necessary.

    @
    void QVtkQuickRenderWindowNode::updateNode()
    {
    //initialize first if necessary
    if(!isInitialized())
    initialize();

    //check if geometry is marked dirty or fbo is empty
    if(((dirtyState() & DirtyGeometry) != 0) || (d_data->glFrameBufferObject == nullptr)) {
        //create new framebuffer and texture (only if size is valid)
        d_data->glFrameBufferObject.reset(validSize() ? new QOpenGLFramebufferObject(d_data->size, QOpenGLFramebufferObject::Depth) : nullptr);
        d_data->fboTexture.reset(validSize() ? new QSGFboTexture(d_data->glFrameBufferObject.get()) : nullptr);
    }
    
    setTexture(d_data->fboTexture.get());
    

    }
    @

    [/quote]

    Thank you very much! What is d_data? I'm very new to OpenGL...

    [quote author="z.emb" date="1360745900"]I think there will be a example regarding rendering a FBO as texture in the next Qt minor release. The guy in the IRC opened a Bug ( https://bugreports.qt-project.org/browse/QTBUG-29548 ) after I asked all my stupid question in the IRC :-)[/quote]

    How I need them now! :-)



  • That is just a private member variable where I store all the variables. See "here":http://en.wikipedia.org/wiki/Pimpl for more information. This has nothing to do with OpenGL.

    You can also add a normal QOpenGLFramebufferObject* and QSGTexture* pointer to your class.



  • Hi z.emb,

    I'm working on this exact problem also.

    Could you tell me what QSGFboTexture is? and also fbo.get() fbo.reset() ?



  • [quote author="Martell Malone" date="1360770728"]
    What about setting the material and the opaque material do you even use them?

    Is there anything else we should know to get this working?
    I seem to be missing something as it keeps crashing.[/quote]

    No I don't use them. I derived my node from QSGSimpleTextureNode, which handles these things for me, I think. I just need to add my own QSGFboTexture and thats it.

    Like I said above, be sure that you don't access any objects in the preprocess() that are also accessed from the GUI thread.
    For debugging reasons you can do the rendering also in the updatePaintNode() method (in my example in the updateNode() method which is called from updatePaintNode() ) to be sure it is not a threading problem.

    Edit:
    [quote author="Martell Malone" date="1360770728"]
    Could you tell me what QSGFboTexture is? and also fbo.get() fbo.reset() ?
    [/quote]

    QSGFboTexture is my subclass for the FBO texture, just implement the pure virtual functions of QSGTexture:

    @
    void QSGFboTexture::setFbo(QOpenGLFramebufferObject* fbo)
    {
    if(d_data->fbo == fbo)
    return;

    d_data->fbo = fbo;
    d_data->firstTimeBound = true;
    

    }

    void QSGFboTexture::bind()
    {
    if(d_data->fbo == nullptr)
    return;

    auto fboTexture = textureId();
    glBindTexture(GL_TEXTURE_2D, fboTexture);
    
    if(d_data->firstTimeBound) {
        updateBindOptions(true);
        d_data->firstTimeBound = false;
    }
    

    }

    bool QSGFboTexture::hasAlphaChannel() const
    {
    return true;
    }

    bool QSGFboTexture::hasMipmaps() const
    {
    if(d_data->fbo == nullptr)
    return false;

    return d_data->fbo->format().mipmap();
    

    }

    int QSGFboTexture::textureId() const
    {
    if(d_data->fbo == nullptr)
    return 0;

    return d_data->fbo->texture();
    

    }

    QSize QSGFboTexture::textureSize() const
    {
    if(d_data->fbo == nullptr)
    return QSize();

    return d_data->fbo->size();
    

    @

    I use the "std::unique_ptr":http://www.cplusplus.com/reference/memory/unique_ptr/ template. Therefore the get() and reset().



  • Thansk for that z.emb

    I was working basing my code from qt3d before which is why I couldn't get it to work
    http://qt.gitorious.net/qt-labs/qt3d/blobs/83fd6411a16f09b81252d4a4398a1968451754c9/src/imports/threed/viewportfbonode_sg.cpp

    Seems you found the solution ;)



  • glPopAttrib and glPushAttrib are invalid functions?

    You can do beginNativePainting within QPainter to access these functions but I'm guessing thats not what you did.

    Im on qt 5.0.1 ANGLE GLES version

    What header did you include?



  • I've made more progress on getting it working on GLES systems.
    My window flickers for a frame and then goes gray.

    Also for that frame the scene is upside down and a texture is missing.

    The last two things that I wish to ask

    1. I noticed
      setFbo isn't called anywhere?

    and also I have two size methods setRect and setSize

    @void ColorNode::setSize(const QSize size)
    {
    if (size == m_size) return;

    m_size = size;
    //m_fbo = NULL;
    //m_texture = NULL;
    
    QSGGeometry::updateTexturedRectGeometry(&m_geometry,
                                                QRectF(0, 0, m_size.width(), m_size.height()),
                                                QRectF(0, 0, 1, 1));
    
    markDirty(DirtyGeometry);
    

    }

    void ColorNode::setRect(const QRectF rect)
    {
    if (m_rect == rect) return

        markDirty(DirtyGeometry);
        m_rect = rect;
    

    }@

    I'm guessing yous implementation is different?
    Could I get a quick look :)

    I think I'm really close



  • Actually, I posted almost the whole implementation already, but here you go: "Node":http://pastebin.com/8CgheNzm . The implementation of the texture and the item are already posted.

    bq. I noticed setFbo isn’t called anywhere?

    You may have noticed that I pass the FBO to the constructor of the texture ;-)

    bq. What header did you include?

    I compiled Qt 5.0.0 with ANGLE deactivated and the -opengl desktop flag, because I use VTK which also uses the native OpenGL system.



  • I need to set the geometry in my setsize or I get nothing in the window.

    @
    QSGFboTexture::QSGFboTexture(QGLFramebufferObject* fbo)
    :QSGTexture()
    {
    m_fbo = fbo;

    // setFbo(fbo);
    if (m_fbo != NULL)
    firstTimeBound = true;

    }
    @

    same as setFbo ;-)

    The problem is that on ANGLE we are using a GLES emulator not OpenGL.
    Those two functions don't exsist.

    What happens to your window when you dont call the push and pop?

    @void ColorNode::preprocess()
    {
    if(m_fbo == NULL) return; //will happen a few times

    QOpenGLContext::currentContext()->functions()->glUseProgram(0);
    m_fbo->bind();
    

    CCDirector::sharedDirector()->mainLoop();

    m_fbo->release();
    

    m_fbo->toImage().save("name.png","png");

    }@



  • Its not working without the push and pop (nothing is rendered, only the background). Unfortunatelly I cannot tell you why, I'm no OpenGL expert.



  • I think I understand why.
    The only problem is GLES doest support those functions so how do we get around them?

    Maybe I should make a new thread to this issue?



  • I found a new way of solving this problem.

    It Involves setting up a separate opengl context and switching between the two before and after binding and releasing the fbo.

    Any idea how to draw correctly for a y inverted fbo.
    On qquickpainteditem we can set it to flip but not for the qsgtexture :/



  • Hey, thanks for your reply. Can you provide some code how you do that?
    In my case, it also seems to be the cleanest way to do the same thing and I wan't to try that. :-)

    For the flipping problem I would naivly just scale (glScale) the y-Axis with -1. But as I mentioned, I'm no OpenGL expert.



  • Hey z.emb

    I used what was on this thread.

    http://qt-project.org/forums/viewthread/25268/

    I just changed this line for the flip y.

    QSGGeometry::updateTexturedRectGeometry(&m_geometry,
    QRectF(0, 0, m_size.width(), m_size.height()),
    QRectF(0, 0, 1, 1));

    to

    QSGGeometry::updateTexturedRectGeometry(&m_geometry,
    QRectF(0, 0, m_size.width(), m_size.height()),
    QRectF(0, 1, 1, -1));//0011 = old y fliped



  • Can you give me some code how you switched the GL-context?



  • Hey Guys,

    Apologies about not replying.
    I didn't realize I got a response.

    I swapped the context at the very beginning by using the code in this thread

    http://qt-project.org/forums/viewthread/25268/

    Hope that helps Ricky, just got your pm.

    Many Thanks
    Martell



  • Hi,

    I still did not get it.
    Could you, please, anyone post a full example about rendering vtk to qtquick?
    I was trying to understand the discussion but I did not get through...

    Best regards,

    radek



  • Hey, I join to this request. Even the simplest example would be helpful.

    Regards,
    rafal


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.