Skip to content
  • Categories
  • Recent
  • Tags
  • Popular
  • Users
  • Groups
  • Search
  • Get Qt Extensions
  • Unsolved
Collapse
Brand Logo
  1. Home
  2. Qt Development
  3. General and Desktop
  4. How to automate user tasks in a Desktop app developed in Qt?
Forum Updated to NodeBB v4.3 + New Features

How to automate user tasks in a Desktop app developed in Qt?

Scheduled Pinned Locked Moved Unsolved General and Desktop
5 Posts 2 Posters 895 Views 3 Watching
  • Oldest to Newest
  • Newest to Oldest
  • Most Votes
Reply
  • Reply as topic
Log in to reply
This topic has been deleted. Only users with topic management privileges can see it.
  • S Offline
    S Offline
    SunnyMax
    wrote on last edited by SunnyMax
    #1

    Hello,
    I would like to know if and how it is possible to automate repetitive user tasks in a business application (on OSX) that had been developed with Qt.

    I am familiar with GUI scripting using Python etc. but this is not what I want to do.
    Sending click commands using xy-coordinates and working with image detection is the least reliable way to automate repetitive tasks.

    Automating application testing neither is what I intend to do.

    What I prefer to achieve is scripting the interactions using the UI elements' properties.
    That's why initially I started using AppleScript and various UI Inspector/Accessibility Inspector tools. They finally revealed classNames indicating that the business App must have been developed in Qt.

    While I can easily edit text fields with Applescript, other UI elements such as button and date picker do not respond to AppleScript commands. A dropdown like view with autosuggestions when typing into the corresponding text field is another example, where I would like to navigate through the items and pick the one matching my criteria defined in the automation routine.

    Since this is my first contact with Qt, though cross reading and searching the Qt documentation, product descriptions as well as the forum, I couldn't find a hint.

    Could you please give me some advice where I could find what is necessary to know to automate interaction with the buttons and other UI elements?

    Many thanks.

    1 Reply Last reply
    0
    • SGaistS Offline
      SGaistS Offline
      SGaist
      Lifetime Qt Champion
      wrote on last edited by
      #2

      Hi and welcome to devnet,

      If I understand you correctly, you are just a user of the application without link to its developers ?

      Interested in AI ? www.idiap.ch
      Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

      S 1 Reply Last reply
      0
      • SGaistS SGaist

        Hi and welcome to devnet,

        If I understand you correctly, you are just a user of the application without link to its developers ?

        S Offline
        S Offline
        SunnyMax
        wrote on last edited by SunnyMax
        #3

        @SGaist yes, exactly, I am just a user of this application. I have no link to the developers. The application is as is.

        1 Reply Last reply
        0
        • SGaistS Offline
          SGaistS Offline
          SGaist
          Lifetime Qt Champion
          wrote on last edited by
          #4

          Then maybe through the accessibility API but I can't guarantee that it will work the way you need it.

          Interested in AI ? www.idiap.ch
          Please read the Qt Code of Conduct - https://forum.qt.io/topic/113070/qt-code-of-conduct

          S 1 Reply Last reply
          0
          • SGaistS SGaist

            Then maybe through the accessibility API but I can't guarantee that it will work the way you need it.

            S Offline
            S Offline
            SunnyMax
            wrote on last edited by SunnyMax
            #5

            After countless hours of research and try and error, here is my outcome for those interested into the same topic:

            Let's first define "automation":
            Hereby I mean that program code (a script) emulates user actions with the application.
            The objective is to free the user from repetitive, boring tasks. This could be for example, data exports on a scheduled basis which are then further analyzed.

            What I am not talking about is:

            • Test automation.
            • automation of industrial processes.

            With this out of the way, I can summarize:

            • Web applications have an automation API based upon the W3C specification of web driver

            • Windows applications have an automation API called UIA3. It supports Win32, Windows Form, WPF.

            A Qt application running on Windows cannot benefit from Window's automation API. The underlying libraries of course cannot reference and interact with QML/QQuick objects.

            If you have a Qt-programmed application that you did not program yourself, there is no way to emulate user interactions!

            There is no API that allows you to address the GUI elements programmatically. The Accessibility API requires that the developers would have invested their time to provide means to connect with accessibility applications. This is very often not the case for obvious reasons.

            In the QT world the rule applies : No access to the source code, no way to automate !

            Squish and Ranorex are commercial tools made for test automation. In theory they could be converted to emulate user interaction with the 3rd party application.

            However, for non-commercial automation projects the following reasons speak against this only possible workaround:

            • the considerable license costs of Squish and Ranorex
            • extreme overhead installing these tools
            • necessity to have a separate package for each Qt version, i.e. if you would like to automate an application A coded in qt 6.2 you need to install another package than for a qt application coded in version 6.4. You get it.
            1 Reply Last reply
            0

            • Login

            • Login or register to search.
            • First post
              Last post
            0
            • Categories
            • Recent
            • Tags
            • Popular
            • Users
            • Groups
            • Search
            • Get Qt Extensions
            • Unsolved