Qt5.7 QVector3D::unproject() mouse picking



  • Hi !
    I would like to "select" an object in my OpenGL world but first of all, I need to find the mouse position in the OpengGL world coordinates and I thought that the QVector3D::unproject() function was the best but I'm not getting what I want : the world coordinates I get don't seem to be right as I created a triangle, one of his corner is (-0.5, -0.5, 0.0) and when I click with the mouse on it, I get (1.96532, -3.93444, 4.93216).
    I tried to change the z coordinate, move the camera ... but I never get close to a working code.
    This is a part of my code so far :

    QMatrix4x4 modelView;
    modelView.setToIdentity();
    modelView.lookAt(m_camera.getPosition(), m_camera.getTarget(), m_camera.getUp());
    
    QMatrix4x4 projection;
    projection.perspective(70.0, width() / height(), 0.1, 120.0);
    
    GLint view[4];
    glGetIntegerv(GL_VIEWPORT, &view[0]);
    
    QVector3D worldPosition = QVector3D(event->pos().x(), event->pos().y(), 0).unproject(modelView, projection, QRect(view[0], view[1], view[2], view[3]));
    qDebug() << worldPosition;
    

    I don't understand what's going wrong. Do you have a clue ?
    Thanks in advance



  • Try and edit following function from my old project: https://github.com/glararan/QEditor/blob/master/src/mapview.cpp#L1175

    I guess based on your description, this is what you're looking for.



  • Thank you for the help, I copy / pasted your code just to try it but I get values from (1.9, -3.8, 4.9) at the top left-hand corner of the window to (2.0, -3.9, 4.8) at the bottom right-hand corner of the window.
    There is your code with my modifications. Do you see what I'm doing wrong ?

        QMatrix4x4 viewMatrix ;
        viewMatrix.lookAt(m_camera.getPosition(), m_camera.getTarget(), m_camera.getUp());
    
        QMatrix4x4 projection;
        projection.perspective(70.0, width() / height(), 0.1, 120.0);
    
        QMatrix4x4 viewportMatrix;
        float w2 = width() / 2.0f;
        float h2 = height() / 2.0f;
    
        viewportMatrix.setToIdentity();
        viewportMatrix.setColumn(0, QVector4D(w2, 0.0f, 0.0f, 0.0f));
        viewportMatrix.setColumn(1, QVector4D(0.0f, h2, 0.0f, 0.0f));
        viewportMatrix.setColumn(2, QVector4D(0.0f, 0.0f, 1.0f, 0.0f));
        viewportMatrix.setColumn(3, QVector4D(w2, h2, 0.0f, 1.0f));
    
        QMatrix4x4 modelMatrix;
        modelMatrix.setToIdentity();
    
        QMatrix4x4 modelViewMatrix = viewMatrix * modelMatrix;
        QMatrix4x4 modelViewProject = projection * modelViewMatrix;
        QMatrix4x4 inverted = viewportMatrix * modelViewProject;
    
        inverted = inverted.inverted();
    
        float posZ;
        float posY = height() - y - 1.0f;
    
        glReadPixels((int)x, (int)posY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &posZ);
    
        QVector4D clickedPointOnScreen(x, posY, 2.0f * posZ - 1.0f, 1.0f);
        QVector4D clickedPointIn3DOrgn = inverted * clickedPointOnScreen;
    
        clickedPointIn3DOrgn /= clickedPointIn3DOrgn.w();
    
        qDebug() << clickedPointIn3DOrgn.toVector3DAffine();
    


  • Try set viewMatrix and projection to setIdentity() but its last thing what can help :D

    I checked your edited code and it seems okay. Where do you call it?



  • I did some experimental researches and I finally discovered the problem, it was the z coordinate.
    I've found the experimental value (0.961635) but I need the law to make the code work if I move the camera.
    This is a new project that I made to see things more clearly :

    #include "mainwindow.h"
    #include <QVector3D>
    #include <QKeyEvent>
    #include <qDebug>
    
    MainWindow::MainWindow(QWidget *parent)
        : QOpenGLWidget(parent)
    {
    }
    
    MainWindow::~MainWindow()
    {
    }
    
    void MainWindow::initializeGL()
    {
        initializeOpenGLFunctions();
        glClearColor(0.7f, 0.7f, 1.0f, 0.0f);
    
        m_program.addShaderFromSourceFile(QOpenGLShader::Vertex, ":/vertexshader.vsh");
        m_program.addShaderFromSourceFile(QOpenGLShader::Fragment, ":/fragmentshader.fsh");
        m_program.link();
    
        m_matrix.setToIdentity();
    
        m_matrix.perspective(70.0, width() / height(), 0.1, 120.0);
        m_matrix.lookAt(QVector3D(0, 0, 5), QVector3D(0, 0, 0), QVector3D(0, 1, 0));
    }
    
    void MainWindow::paintGL()
    {
        glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
    
        m_program.bind();
    
        m_program.setUniformValue("mvp", m_matrix);
    
        GLfloat vertices[] = {
            0.0f, 1.0f, 0.0f,
            -0.5f, -0.5f, 0.0f,
            0.5f, -0.5f, 0.0f
        };
    
        glVertexAttribPointer(m_program.attributeLocation("vertexPosition_modelspace"), 3, GL_FLOAT, GL_FALSE, 0, vertices);
    
        glEnableVertexAttribArray(0);
    
        glDrawArrays(GL_TRIANGLES, 0, 3);
    
        glDisableVertexAttribArray(0);
    
        m_program.release();
    }
    
    void MainWindow::mousePressEvent(QMouseEvent *event)
    {
        QMatrix4x4 viewMatrix ;
        viewMatrix.setToIdentity();
        viewMatrix.lookAt(QVector3D(0, 0, 5), QVector3D(0, 0, 0), QVector3D(0, 1, 0));
    
        QMatrix4x4 projection;
        projection.setToIdentity();
        projection.perspective(70.0, width() / height(), 0.1, 120.0);
    
        QMatrix4x4 viewportMatrix;
        float w2 = width() / 2.0f;
        float h2 = height() / 2.0f;
    
        viewportMatrix.setToIdentity();
        viewportMatrix.setColumn(0, QVector4D(w2, 0.0f, 0.0f, 0.0f));
        viewportMatrix.setColumn(1, QVector4D(0.0f, h2, 0.0f, 0.0f));
        viewportMatrix.setColumn(2, QVector4D(0.0f, 0.0f, 1.0f, 0.0f));
        viewportMatrix.setColumn(3, QVector4D(w2, h2, 0.0f, 1.0f));
    
        QMatrix4x4 modelMatrix;
        modelMatrix.setToIdentity();
    
        QMatrix4x4 modelViewMatrix = viewMatrix * modelMatrix;
        QMatrix4x4 modelViewProject = projection * modelViewMatrix;
        QMatrix4x4 inverted = viewportMatrix * modelViewProject;
    
        inverted = inverted.inverted();
    
        float posZ;
        float posY = height() - event->pos().y() - 1.0f;
    
        glReadPixels((int)event->pos().x(), (int)posY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &posZ);
    
        qDebug() << 2.0f * posZ - 1.0f;
        QVector4D clickedPointOnScreen(event->pos().x(), posY, 0.961635, 1.0f);
        QVector4D clickedPointIn3DOrgn = inverted * clickedPointOnScreen;
    
        clickedPointIn3DOrgn /= clickedPointIn3DOrgn.w();
    
        qDebug() << clickedPointIn3DOrgn.toVector3DAffine();
    }
    
    


  • After some testing, it turned out that I need the projected z coordinate to unproject the x and y but in order to get it, I need the unprojected x and y ... it's an infinite loop !
    Here's the code I came up with :

    QMatrix4x4 viewMatrix ;
    viewMatrix.lookAt(m_camera.getPosition(), m_camera.getTarget(), m_camera.getUp());
    
    QMatrix4x4 projection;
    projection.perspective(70.0, width() / height(), 0.1, 120.0);
    
    QVector3D Z(0, 0, 0); // instead of 0 for x and y i need worldPosition.x() and worldPosition.y() ....
    Z = Z.project(viewMatrix, projection, QRect(0, 0, width(), height()));
    
    QVector3D worldPosition = QVector3D(x, height() - y, Z.z()).unproject(viewMatrix, projection, QRect(0, 0, width(), height()));
    qDebug() << worldPosition;
    


  • I was searching for a way the pass few days. This reference is the most detailed I have seen.
    Long story short, screen coordinates (x, y, 0) will transfer into the near field z value, and (x, y, 1) will transfer into far field z value. If you want the values in between a specific position, linear interpolation will help.


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.