Converting screen coordinates back to world coordinates for OpenGL picking
I am working on my OpenGL picking using ray casting. I am stuck at converting screen coordinates (mouse clicked 2D point) to 3D world coordinates.
I was trying to use QVector3D::unproject(), not sure if this is the correct way in Qt to do this. The question is how is the Z value I should use with the clicked X and Y for the conversion. From below code snippet you can see I get the depth value using glReadPixels with GL_DEPTH_COMPONENT even though this enum param is not supported by OpenGL ES2.0. The depth value I retrieved is always clamped into [0 1] and the resulted 3D coordinates is also not correct. Can anyone help me out with this? Thanks.
// read the depth value posZ of the clicked point (posX, posY) on screen glReadPixels(posX, posY, 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &posZ); // calculate viewmodel matrix QMatrix4x4 modelViewMatrix = viewMatrix * modelMatrix; // get the viewport rectangle const qreal retinaScale = devicePixelRatio(); QRect viewPortRect(0, 0, width() * retinaScale, height() * retinaScale); // unproject to 3D QVector3D clickedPointIn3DWorld = QVector3D(posX, posY, posZ).unproject(modelViewMatrix, projectionMatrix, viewPortRect);