Capturing the mouse inside a QOpenGLWidget

[mrjbom]

Hello.

I have a QOpenGLWidget widget, I use the mouse pointer to control the camera and I would like to be able to capture the mouse inside the widget for easier control.

How can I do this?

[Chris Kawa]

The way I usually do it is something like this:

• Hide native cursor (i.e. set it to Qt::BlankCursor) and move it with setPos() to center of the widget
• In mouseMoveEvent() calculate the movement delta (pos() minus widget center)
• Update a "virtual" cursor position (some local QPoint variable) using that delta and clamp it to the dimensions of your widget
• Draw the "virtual" cursor as part of your rendering or as a separate overlay widget above yours.
• Move native cursor back to the center of the widget ignoring move event

This has the benefit of not having to deal with what happens when you get to the edge of the screen (because you never will) and also allows for nice features like cursor wrapping (when you move past left border the cursor is moved to the right etc) which are common in 3D apps.

You could try to do it with the native cursor directly, but it always ends up shaky and twitchy, especially near borders.

[mrjbom]

This post is deleted!

[mrjbom]

@Chris-Kawa
I'm trying to implement your suggestion, but trying to keep the cursor within the widget makes it impossible to move the camera, because there are new calls to the move handler functions.

void MainOpenGLWidget::mousePressEvent(QMouseEvent *event)
{
    //lock cursor inside widget
    this->cursor().setPos(this->parentWidget()->mapToGlobal(this->pos()));
    mouseLastPositon = QPoint(0, 0);
    if(!mouseCaptured) {
        this->setCursor(Qt::BlankCursor);
        savedWindowTitle = this->nativeParentWidget()->windowTitle();
        mouseCaptured = true;
    }
    this->nativeParentWidget()->setWindowTitle("Press Alt to release cursor");

    virtualMousePosition = QPoint(0, 0);
    event->setLocalPos(virtualMousePosition);
    sceneManager->callMousePressEventHandler(event);
}

void MainOpenGLWidget::mouseReleaseEvent(QMouseEvent *event)
{
    event->setLocalPos(virtualMousePosition);
    sceneManager->callMouseReleaseEventHandler(event);
}

void MainOpenGLWidget::mouseMoveEvent(QMouseEvent *event)
{
    //"Note that the returned value(event::button()) is always Qt::NoButton for mouse move events."

    float xpos = event->pos().x();
    float ypos = event->pos().y();

    float xoffset = xpos - mouseLastPositon.x();
    float yoffset = mouseLastPositon.y() - ypos;

    mouseLastPositon = event->pos();

    virtualMousePosition += QPoint(xoffset, -yoffset);
    event->setLocalPos(virtualMousePosition);

    sceneManager->callMouseMoveEventHandler(event);
    this->cursor().setPos(this->parentWidget()->mapToGlobal(this->pos()));
}

[Chris Kawa]

Couple issues here.

First - you're moving the cursor to the upper left corner of the widget instead of the center like I mentioned. This could prove problematic e.g. if you maximize your widget you won't be able to move up or left because the cursor won't go outside of the screen. I'd strongly advise to use the center, so simply:

QCursor::setPos(mapToGlobal(rect().center()));

btw. QCursor::setPos() is static so you don't need to use this->cursor().

Second - the extra event when you're re-centering the cursor. Like I said you need to ignore it for your virtual position calculations and not reposition the cursor again. You could introduce some extra flag for that but really the easiest way is to simply check in the move event handler if the new position is the center of your widget. The only way it can happen is by that artificial move. All "real" moves will be around that point with non-zero delta, so:

if (event->pos() != rect().center())
{
    ...
}

Third - you might not notice it right away but there could be subtle issues with the way you hijack the events. One example is that enter/leave events might stop working properly. My advice would be to pass the necessary params to your scene manager as copies and always call the base xxxEvent implementations with unmodified event parameter so that all the window handling works correctly.

[mrjbom]

@Chris-Kawa
I changed the code
however, I still can't move the camera for the same reason as before

void MainOpenGLWidget::mousePressEvent(QMouseEvent* event)
{
    QMouseEvent eventCopy = *event;
    QCursor::setPos(mapToGlobal(this->rect().center()));
    qInfo() << event->pos();
    //lock cursor inside widget
    if (event->pos() != this->rect().center())
    {
        mouseLastPositon = this->rect().center();
        if(!mouseCaptured) {
            this->setCursor(Qt::BlankCursor);
            savedWindowTitle = this->nativeParentWidget()->windowTitle();
            mouseCaptured = true;
        }
        this->nativeParentWidget()->setWindowTitle("Press Alt to release cursor");

        virtualMousePosition = QPoint(0, 0);
        eventCopy.setLocalPos(virtualMousePosition);
        sceneManager->callMousePressEventHandler(eventCopy);
    }
}

void MainOpenGLWidget::mouseReleaseEvent(QMouseEvent* event)
{
    QMouseEvent eventCopy = *event;
    QCursor::setPos(mapToGlobal(this->rect().center()));
    sceneManager->callMouseReleaseEventHandler(eventCopy);
}

void MainOpenGLWidget::mouseMoveEvent(QMouseEvent* event)
{
    //"Note that the returned value(event::button()) is always Qt::NoButton for mouse move events."

    QMouseEvent eventCopy = *event;
    if (event->pos() != this->rect().center())
    {
        float xpos = event->pos().x();
        float ypos = event->pos().y();

        float xoffset = xpos - mouseLastPositon.x();
        float yoffset = mouseLastPositon.y() - ypos;

        mouseLastPositon = event->pos();

        virtualMousePosition += QPoint(xoffset, -yoffset);
        eventCopy.setLocalPos(virtualMousePosition);

        sceneManager->callMouseMoveEventHandler(eventCopy);
        QCursor::setPos(mapToGlobal(this->rect().center()));
    }
}

In addition, I encounter some twitching at the beginning of the mouse movement(if I press the button while the mouse is moving)

[Chris Kawa]

You complicated it a bit with storing that last position and you're still not calling base implementation.
Here's a simple working example. Since you're using OpenGL Just remove calls to update() and paintEvent and it should work fine.

class Test : public QWidget
{
    QPoint virtual_pos;
public:
    void mousePressEvent(QMouseEvent* evt) override;
    void mouseReleaseEvent(QMouseEvent* evt) override;
    void mouseMoveEvent(QMouseEvent* evt) override;
    void paintEvent(QPaintEvent* evt) override;
};

void Test::mousePressEvent(QMouseEvent* evt)
{
    virtual_pos = evt->pos();
    setCursor(Qt::BlankCursor);
    QCursor::setPos(mapToGlobal(rect().center()));
    update();
    // sceneManager->callMousePressEventHandler(virtual_pos);
    QWidget::mousePressEvent(evt);
}

void Test::mouseReleaseEvent(QMouseEvent* evt)
{
    QCursor::setPos(mapToGlobal(virtual_pos));
    setCursor(Qt::ArrowCursor);
    // sceneManager->callMouseReleaseEventHandler(virtual_pos);
    QWidget::mouseReleaseEvent(evt);
}

void Test::mouseMoveEvent(QMouseEvent* evt)
{
    if (evt->buttons() & Qt::LeftButton && evt->pos() != rect().center())
    {
        virtual_pos += (evt->pos() - rect().center());

        //uncomment if you want wrap behavior
        //virtual_pos.setX((virtual_pos.x() + width()) % width());
        //virtual_pos.setY((virtual_pos.y() + height()) % height());

        virtual_pos.setX(qBound(0, virtual_pos.x(), width()));
        virtual_pos.setY(qBound(0, virtual_pos.y(), height()));
        QCursor::setPos(mapToGlobal(rect().center()));
        update();
        // sceneManager->callMouseMoveEventHandler(virtual_pos);
    }
    QWidget::mouseMoveEvent(evt);
}

void Test::paintEvent(QPaintEvent* evt)
{
    QWidget::paintEvent(evt);
    QPainter p(this);
    p.drawEllipse(virtual_pos, 5, 5);
}

[mrjbom]

Now I can move the mouse correctly, but this does not save me from a strange jerk at the beginning.
If I move the mouse over the drawing area and press the left mouse button while it is moving, the camera twitches.
I can't understand what this is related to.

void MainOpenGLWidget::mousePressEvent(QMouseEvent* event)
{

    virtualMousePosition = event->pos();
    setCursor(Qt::BlankCursor);
    QCursor::setPos(mapToGlobal(rect().center()));
    QMouseEvent eventCopy(event->type(), virtualMousePosition, event->button(), event->buttons(), event->modifiers());
    sceneManager->callMousePressEventHandler(eventCopy);
    QWidget::mousePressEvent(event);
}

void MainOpenGLWidget::mouseReleaseEvent(QMouseEvent* event)
{
    //QMouseEvent eventCopy = *event;
    //QCursor::setPos(mapToGlobal(this->rect().center()));
    //sceneManager->callMouseReleaseEventHandler(eventCopy);

    QCursor::setPos(mapToGlobal(virtualMousePosition));
    setCursor(Qt::ArrowCursor);
    QMouseEvent eventCopy(event->type(), virtualMousePosition, event->button(), event->buttons(), event->modifiers());
    sceneManager->callMouseReleaseEventHandler(eventCopy);
    QWidget::mouseReleaseEvent(event);
}

void MainOpenGLWidget::mouseMoveEvent(QMouseEvent* event)
{
    //"Note that the returned value(event::button()) is always Qt::NoButton for mouse move events."

    if (event->buttons() & Qt::LeftButton && event->pos() != rect().center())
    {
        virtualMousePosition += (event->pos() - rect().center());

        //uncomment if you want wrap behavior
        //virtual_pos.setX((virtual_pos.x() + width()) % width());
        //virtual_pos.setY((virtual_pos.y() + height()) % height());

        virtualMousePosition.setX(qBound(0, virtualMousePosition.x(), width()));
        virtualMousePosition.setY(qBound(0, virtualMousePosition.y(), height()));
        QCursor::setPos(mapToGlobal(rect().center()));
        QMouseEvent eventCopy(event->type(), virtualMousePosition, event->button(), event->buttons(), event->modifiers());
        sceneManager->callMouseMoveEventHandler(eventCopy);
    }
    QWidget::mouseMoveEvent(event);
}

Example:

I move the mouse from right to left and while moving, I click on the left mouse button, continue to move to the left.

As a result, I get this incorrect output:

press real mouse position QPoint(626,336)
press virtual mouse position QPoint(626,336)
move real mouse position QPoint(625,336)
move virtual mouse position QPoint(719,332)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(717,332)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(716,332)
move real mouse position QPoint(356,340)
move virtual mouse position QPoint(713,332)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(712,332)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(710,332)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(709,332)
move real mouse position QPoint(356,340)
move virtual mouse position QPoint(706,332)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(704,332)
move real mouse position QPoint(358,339)
move virtual mouse position QPoint(703,331)
move real mouse position QPoint(356,339)
move virtual mouse position QPoint(700,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(699,330)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(697,330)
move real mouse position QPoint(355,340)
move virtual mouse position QPoint(693,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(692,330)
move real mouse position QPoint(356,340)
move virtual mouse position QPoint(689,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(688,330)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(686,330)
move real mouse position QPoint(356,340)
move virtual mouse position QPoint(683,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(682,330)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(680,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(679,330)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(677,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(676,330)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(674,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(673,330)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(671,330)
move real mouse position QPoint(356,340)
move virtual mouse position QPoint(668,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(667,330)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(665,330)
move real mouse position QPoint(356,340)
move virtual mouse position QPoint(662,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(661,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(660,330)
move real mouse position QPoint(357,340)
move virtual mouse position QPoint(658,330)
move real mouse position QPoint(356,340)
move virtual mouse position QPoint(655,330)
move real mouse position QPoint(358,340)
move virtual mouse position QPoint(654,330)

These coordinates of the virtual cursor movement are incorrect, because the first move must be less than press.
The mouse also twitches to the right(as I move to the left), this can be seen from the coordinates.

[Chris Kawa]

Hm, the problem here is that we're interacting with the OS, and this adds some latency. During that time OS does its thing and keeps sending events that piled up, so what happens here is

press real mouse position QPoint(626,336) // Calling setPos(), this gets scheduled in the OS
move  real mouse position QPoint(625,336) // This was already queued after above, just waited for delivery
...                                       // OS generated a mouse move event for our setPos() with x = 358
...                                       // Because mouse was moving (delta x=-1) OS aggregated two close move events into one and sent x=357
move  real mouse position QPoint(357,340) // This is the aggregate of setPos() and the movement in flight
move  real mouse position QPoint(358,340) // This is the next setPos() result
move  real mouse position QPoint(356,340) // This is the next "real" movement
move  real mouse position QPoint(358,340) // This is the next setPos() result
move  real mouse position QPoint(357,340) // This is the next "real" movement
...

What you'd ideally want here is to ignore all te moves after press until the first "real" movement. Unfortunately there's no API to do that in Qt - no way to tell which move event corresponds to which setPos() call and it might be that OS aggregates some moves so there's no direct translation.

I guess what you could try to do is implement sort of a "dead zone" after the press, i.e. on press set some flag, then in move event clear it only when position gets close enough (within one or two points) to center and ignore the move until that flag is cleared.
One problem with this would be if you actually started within that zone around the center, but in this case the jump wouldn't be bigger than that one or two points, so shouldn't be visible.

[mrjbom]

@Chris-Kawa Okay, I implemented your idea with the zone and got rid of the jumps.
But unfortunately all these manipulations with via QCursor:: setPos lead to twitching of the camera.

I believe that due to the lack of the necessary APIs in Qt, it is not possible to implement mouse capture for my task.
Thanks for the help.