Inspiration for my Transparent Overlay Program

[opus]

Hey guys,
I am developing a transparent (full-screen) always-on-top overlay. Let us say its name is TRA. It should be able to display tooltips in a specific region of the display. I am developing with C++ and Qt6.

I am new to Windows and its windows system/manager as well as Qt6, so I tried a lot, but now I guess I need inspiration from someone more experienced... I work on two scenarios and maybe someone has an idea how to solve one of them:

1. Non full-screen overlay: I finished my work making TRA just for the specific region. The problem is that TRA is always on top, which is good, but when I click outside the specific region and thus TRA, TRA is not focused anymore and do not show any tooltips until its focussed again. Is it possible to have TRA always reacting to show tooltip independent from any actions outside of TRA? I use this currently:

TransparentWindow::TransparentWindow(QWidget* parent) : QWidget(parent){

    //...

    setWindowFlags(Qt::FramelessWindowHint | Qt::WindowStaysOnTopHint);
    setAttribute(Qt::WA_TranslucentBackground);
    setAttribute(Qt::WA_NoSystemBackground);
    setAttribute(Qt::WA_TransparentForMouseEvents, false);

    //...

    show();
}

2. Full-screen overlay: To overcome the window-focus problem, I decided to try a full-screen variant of it which needs to pass all mouse clicks, so, the behavior would be like without executing TRA. Tooltips are working fine because with every click the TRA stays in focus. Now, I need to pass the clicks to all the over windows/programs in Windows behind TRA. I found the exstyles from the Windows.h, so I build something like that in Qt:

void TransparentWindow::mousePressEvent(QMouseEvent *event){

    QPoint pos = event->globalPos();
    if(event->button() == Qt::LeftButton){
        sendClick(pos, Mouse::LEFT_DOWN);
    } else if(event->button() == Qt::RightButton){
        sendClick(pos, Mouse::RIGHT_DOWN);
    }
}

void TransparentWindow::mouseReleaseEvent(QMouseEvent *event){

    QPoint pos = event->globalPos();
    if(event->button() == Qt::LeftButton){
        sendClick(pos, Mouse::LEFT_UP);
    } else if(event->button() == Qt::RightButton){
        sendClick(pos, Mouse::RIGHT_UP);
    }
}

void TransparentWindow::setWindowTransparent(){
    m_exStyle |= WS_EX_LAYERED | WS_EX_TRANSPARENT;
    SetWindowLong(m_hwnd, GWL_EXSTYLE, m_exStyle);
    SetLayeredWindowAttributes(m_hwnd, RGB(0, 0, 0), 255, LWA_ALPHA);
    Sleep(100); // 50 also works
}

void TransparentWindow::unsetWindowTransparent(){
    m_exStyle &= ~WS_EX_LAYERED & ~WS_EX_TRANSPARENT;
    SetWindowLong(m_hwnd, GWL_EXSTYLE, m_exStyle);
    SetLayeredWindowAttributes(m_hwnd, RGB(0, 0, 0), 255, LWA_ALPHA);
}

void TransparentWindow::sendClick(const QPoint &pos, Mouse mouse){

    setWindowTransparent();

    POINT point;
    point.x = static_cast<LONG>(pos.x());
    point.y = static_cast<LONG>(pos.y());

    HWND hwnd = WindowFromPoint(point);

    if (hwnd != nullptr) {
        INPUT input;

        input.type = INPUT_MOUSE;
        input.mi.dx = 0; 
        input.mi.dy = 0; 
        input.mi.mouseData = 0;
        input.mi.time = 0;
        input.mi.dwExtraInfo = 0;

        switch (mouse)
        {
        case Mouse::LEFT_DOWN:
            input.mi.dwFlags = MOUSEEVENTF_LEFTDOWN; 
            m_mouseProcessing = true; //this needs to be set (?) otherwise the sendclick(), "clicks" TRA again?
            break;
        case Mouse::LEFT_UP:
            input.mi.dwFlags = MOUSEEVENTF_LEFTUP; 
            m_mouseProcessing = false;
            break;
        case Mouse::RIGHT_DOWN:
            input.mi.dwFlags = MOUSEEVENTF_RIGHTDOWN; 
            m_mouseProcessing = true; //this needs to be set (?) otherwise the sendclick(), "clicks" TRA again?
            break;
        case Mouse::RIGHT_UP:
            input.mi.dwFlags = MOUSEEVENTF_RIGHTUP; 
            m_mouseProcessing = false;
            break;
        }

        // SetCursorPos(point.x, point.y);

        SendInput(1, &input, sizeof(INPUT));

    } else {
        qDebug() << "No Window found.";
    }

    unsetWindowTransparent();
}

Technically it works, but it is not pretty and definitely needs some work on these aspects:

• I guess there is a race condition: Without the Sleep(100) the clicks are not passed through successfully.
• The recreation(?) of the exstyles (plus the sleep?) makes the transparent window full black for a very short time. This need to be removed.
• m_mouseProcessing needs to be set otherwise the sendclick(), "clicks" TRA again?

--

Does anyone have an elegant ways to solve either of the scenarios?