Intercepting mouse events and getting element information ? ERROR in Qt

learnist

I am able to intercept mouse events, but the desired effect i want , is that when i click on an element(button/listitem/combobox) using mouse cursor, i should get its information(Name/AutomationID) in my UI ?

How can i achieve this ?

To achieve this, i have tried some code but it gives me errors, here is my files , i guess i need a bit of change in my mainwindow.cpp mouseproc function, i hope i am clear, have provided comments in all parts where needed, Thanks.

mainwindow.cpp

#include "mainwindow.h"
#include "ui_mainwindow.h"
using namespace std;

 IUIAutomation* automation = NULL;  //Here are the global variables, which i am gonna use if my Wparam == "WM_RBUTTONDOWN"
 IUIAutomationElement* elem = NULL;   
 POINT mousePt;
 BSTR elemName = NULL;
 //BOOL stat = InitializeUIAutomation(&automation);


 BOOL InitializeUIAutomation(IUIAutomation** automation)
 {
     CoInitialize(NULL);
     HRESULT hr = CoCreateInstance(__uuidof(CUIAutomation), NULL,
         CLSCTX_INPROC_SERVER, __uuidof(IUIAutomation),
         (void**)automation);
     return (SUCCEEDED(hr));
 }
 
 MainWindow& MainWindow::instance() //instance 
 {
     static MainWindow _instance;
     return _instance;
 }

MainWindow::MainWindow(QWidget* parent)
    : QMainWindow(parent)
    , ui(new Ui::MainWindow)
{
    ui->setupUi(this);

    //added here
     HINSTANCE hInstance = GetModuleHandle(NULL);

    // Set hook
    mouseHook = SetWindowsHookEx(WH_MOUSE_LL, mouseProc, hInstance, 0);
    // Check hook is correctly
    if (mouseHook == NULL)
    {
        qWarning() << "Mouse Hook failed";
    }
}

void MainWindow::closeEvent(QCloseEvent* event)  // function executed when window is closed.
{
    // Code for destroy
    exContext.destroy();
    //automation->Release();
    CoUninitialize();

    mouseThread->Stop = true;
    mouseThread->quit();

    exContext.pUIAutomation = NULL;
}

MainWindow::~MainWindow()
{
    delete ui;
}

LRESULT CALLBACK MainWindow::mouseProc(int Code, WPARAM wParam, LPARAM lParam)
{
    Q_UNUSED(Code)
        auto& ms = *(const MOUSEHOOKSTRUCT*)lParam;
        MOUSEHOOKSTRUCT* pMouseStruct = (MOUSEHOOKSTRUCT*)lParam;
    if (pMouseStruct != nullptr) 
    {  
        if (wParam == WM_RBUTTONDOWN)
        {
            IUIAutomation* automation = NULL;

                    GetCursorPos(&mousePt);
                     HRESULT hr = automation->ElementFromPoint(mousePt, &elem);  //Getting error here 
                    if (SUCCEEDED(hr) && elem != NULL)
                    {
                        elem->get_CurrentName(&elemName);
                        std::wstring ws(elemName, SysStringLen(elemName));
                        std::wcout << ws << std::endl;
                        QString testing = QString::fromStdWString(ws);
                        qDebug() << testing;
                    }
                    SysFreeString(elemName);
                    elem->Release();
        }
    
        emit instance().mouseEvent();
    }
    // After that you need to return back to the chain hook event handlers
    return CallNextHookEx(NULL, Code, wParam, lParam);
}

mainwindow.h

#pragma once
#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QThread>
#include <QtWidgets/QMainWindow>
#include <QtWidgets/qlistwidget.h>

QT_BEGIN_NAMESPACE
namespace Ui { class MainWindow; }
QT_END_NAMESPACE


class MainWindow : public QMainWindow
{
    Q_OBJECT                      
    Q_DISABLE_COPY(MainWindow) //added

public:
    MainWindow(QWidget* parent = 0);
    ~MainWindow();                   

    static MainWindow &instance //Here is the instance
    // Static method that will act as a callback-function
   static LRESULT CALLBACK mouseProc(int Code, WPARAM wParam, LPARAM lParam);

private slots:

public:
    BOOL InitializeUIAutomation(IUIAutomation** automation); //function

    Ui::MainWindow* ui; // pointing to UI class

signals:
    // The signal, which will report the occurrence of an event
   void mouseEvent();
   
private:  
    HHOOK mouseHook;    // hook handler
};
#endif // MAINWINDOW_H

main.cpp

#include "mainwindow.h"
#include <qdebug.h>
#include <UIAutomation.h>
#include <QtWidgets/qapplication.h>  

int main(int argc, char** argv)
{
    QApplication a(argc, argv);
    MainWindow w;
  
    QObject::connect(&MainWindow::instance(), &MainWindow::mouseEvent, //Connecting instance with mouseEvent
        []() {
            qDebug() << "Mouse Event";
        });
        
    w.show();  
    return a.exec();

}

SGaist

Hi,

QObject based classes cannot be copied.
Static QObject is also wrong. QApplication should be the first QObject created as it setups the internal state to make things work.

learnist

@SGaist im sorry for my noob question, but i don't understand , where did i go wrong? can u tell me file name and probably line please ?

SGaist

Your singleton instance of MainWindow, that is one of the issue.

JonB

@learnist
Further to @SGaist: why do you try to have MainWindow::instance() returning a static MainWindow while you have a local MainWindow w;?