Feature request: Scientific method for UI design. "UX Analytics Studio"
-
Hi Qt (community),
in this message I want to communicate a feature request "UX Analytics Studio" to improve usability of GUIs by using scientific method (data science and machine learning) (Section 1).
Later, I also describe a technique which can be used to collect labeled data, tag workflow steps, from UI events (Section 2).
The technique I describe can be used to train supervised machine learning models to identify workflows/workflow steps from unlabeled data.
Unsupervised methods, clustering?, can be used to make guesses of workflows from raw input event data (not 100% sure about this one, not expert in ML).
The idea behind identifying workflow steps from events is to understand via data on how UI and UI changes affect efficiency of workflows for users, statistically.1. Idea
More precisely, this feature request closely follows the ideas of the book "Process Mining", by Wil van der Aalst,
and kind of (in the long run, if data is collected analyzed from multiple users and multiple UIs) digitalizes "Action research" from social sciences.
I want to apply the Process Mining technique to- Identify at which workflow step a user is at based on Qt input events
- Identify new workflows/techniques from user Qt input events
Perhaps the best way to understand what I'm looking for is by trying to understand how people use Blender to create things.
A user first, if not abstractly brainstorming, has some idea on what the user wants to create.
A user then spits construction of the idea into steps (in the same way as house is built in steps). These steps create high level workflow.
Then all these steps are split into smaller steps, a lower workflow, and this splitting is continued all the way down to basic operations.
These basic operations are then communicated to the system/program using UI input events.Now, I want to record the input events of users (event logger), perhaps selectively (no passwords), to create estimates via data science and machine learning
of where the user is at the workflows (all levels). The reasons I want to do this are the following- Just like sports, reading, and writing a forum post are mostly about technique (not so much about strength), I believe that using UI efficiently is also mostly about technique.
- I want to understand and communicate the techniques used by the world bests, most efficient users, to other users, to improve their efficiency. Perhaps via interactive chatbots/coaches (not important to get the idea).
- I want to monitor that no user is left behind, in terms of techniques, so that they get the most out of the UI.
- I want to study statistically, like A/B testing, on how changes to UI affect workflow efficiencies.
I want Qt to include capability to address the things I described to their portfolio, "UX Analytics Studio".
I believe that this capability has to be done at Qt level because including an event logger to every project using Qt is unrealistic and, in my opinion,
might look like one tries to insert vulnerabilities to programs to collect passwords.2. Labeled data and supervised machine learning
Checklist/workflow diagram smartphone/tablet app to tag input event time series of users (tags input events of external Qt app, not the smartphone/tablet app).
A user selects a workflow diagram to execute. And the first step is marked as active.
Then the user moves to next workflow step by clicking on it.
The input events in Qt app are labeled to the state that was active in the smartphone/tablet app.3. Closing Remarks
In the above I concentrated on traditional UI event: mouse and keyboard.
I think it would a good idea to take VR and AR into account as well.
By this I mean following how users eye pupils move and where pupils look at while using the UI (also hands, feet, ...).
Perhaps later add brain imaging options (neural activity)? For example, take a look at the video on Tempest fighter
https://www.youtube.com/watch?v=Rx-Rey3bA-MCritique, improvements, and further brainstorming welcome.
Best Regards,
Tomi Pannila -
And make it easier for 3rd party data mining, no? computer use privacy is already a joke and your proposal simple aims to make is easier for people to gather and organize info that in most cases is none of their business.
-
Hi Kent-Dorfman,
user data privacy is indeed an important topic.
All useful data mining tools make it easier for any party to mine data.
This idea is no exception.The idea aims to standardize and improve the data collection and analysis process for Qt GUIs.
Organizations and companies prefer to make decisions based on data. See for example
https://wiki.mozilla.org/Data_CollectionSee also Hotjar for web https://www.hotjar.com/ .
I would like to see something similar for Qt, but more comprehensive.Best regards,
Tomi Pannila -
Hi and welcome to devnet,
You are here on a user forum, if you want it discuss that kind of feature you should open feature request. Note that Qt Creator already does some aggregation if enabled.
There's also the KUserFeedback module that might interest you.
