User Customizable Touch Screen Interface.

cdw3423

I am working on a project that will use a Raspberry Pi with a 7" touch screen to control an external device. I'm looking for input and possible example apps that I could use to get started. I don't plan on doing a lot of other embedded Qt projects, so I'm hoping I can find examples that can get me going pretty quickly.

I need to have the interface configured by the end user. All the app will be doing is responding to touch events by looking up a list of commands to send to an external device based on what the user touched. So there isn't any logic involved other than knowing what the user pushed and finding the correct list of commands.

My first thought is the user would supply a set of full screen image files along with corresponding config files. The config file would define buttons by indicating the upper left and lower right corner pixel coordinates and then give a list of commands to execute. They would be able to swipe left or right to switch between different pages of buttons. In this case the app would just need to call a function that will process each touch event passing it the x,y location of the touch event and the name of the page being displayed.

The second option I am considering is having the user (in the config file) a number of rows and columns of buttons and then a have a set of icon image files using spreadsheet style notion. So the config file would be called page01.conf and then there would be a set of images called page01btnA1.xyz, page 01btnB2 .....
In this case the code would need to generate a grid of buttons with a number of columns and rows from the config file and then load the icon images as background images for each button.

Thanks for the input.

Chris W

SGaist

HI and welcome to devnet,

What kind of commands would your users have at hand ?

cdw3423

The commands are specific to the external equipment that is being controlled and don't really matter to the interface. They will be saved in a simple text file and when the button associated with those commands is pressed all this app is going to do is send them out over a UART serial connection. The text file will be manually generated on a PC by the end users and transferred to the Raspberry Pi.

If I go with option one, the user will also have to supply a 800 x 480 images that each represents a page of buttons. The Touch Screen is 500 x 480. If I go with the second option where they supply icon files for the buttons that are then mapped to actual Qt button objects in a grid, I will probably have Qt scale the icons to fit the size of the button. This will be handy if I decide to make a mobile version that uses a Bluetooth serial adapter.

SGaist

So you have some sort of editor on the desktop machine to create files to generate the UI on the Pi ?

cdw3423

Yes but it's not special software it is just any text editor to create the config file and any image editor to create/edit either button icons or button page images. So the Qt app will need to read the config file, display either a single image full screen and respond to touch events or read icon files into a grid of Qt buttons objects and then each button will execute code to send the commands from the same config file out the serial port

SGaist

Then it really looks like you should provide an editor that generates the configuration files the way your application expects them rather than rely only user input without any validation.

cdw3423

Maybe, but that has nothing to do with Qt or how I should design the app. I'm asking for input on how to use Qt. The way my config files are maintained is not relevant. What I need help with is how to get started with the GUI in Qt.

SGaist

Well, partly it does, because you could be generating QtQuick controls out of your configuration for your application to load for example. Thus it could be mainly a "loader" that handles these "pages".

Otherwise you'd need a parser to generate dynamically the interface.

In any case, you should take a look at QtQuick and the Graphics View framework to see which one might fit better your need.

I'd rather not start with custom static images. You would have to know the place of each and every button, check for every touch whether it's in the geometry of any button and the user wouldn't have any visual feed back that there's something happening when he presses one of theses buttons.

cdw3423

I took a look at QtQuick and created a simple qml app with code listed below and the image I used to represent all 20 buttons.

So I have a few questions.
How can I load the qml file at run time and also the image file. The only way I could get the image to show up at run time was to manually edit the qml.qrc file and add <file>Buttons.png</file>. Before I did that, the image would show up in the QtCreator design view but would not show up when I ran the app. As you can see, there is user feedback when the mouse is pressed. I'm assuming I can use similar code for touch events. Also I need to figure out how to have a swipe action switch between images.
Right now I am doing this all on a windows PC, it sounds like the best way to build this for the Raspberry pi is to install linux on a PC and build it with Qt on linux.

MainForm.ui.qml

import QtQuick 2.6

Rectangle {
    property alias mouseArea: mouseArea
    width: 500
    height: 400
    property alias repeater: repeater

    Image {
        id: image
        anchors.fill: parent
        source: "Buttons.png"
    }

    MouseArea {
        id: mouseArea
        anchors.fill: parent
        Grid {
            id: grid
            rows: 4
            columns: 5
            anchors.fill: parent
            Repeater {
                id:repeater
                model: 20
                Rectangle{
                    width: 100
                    height: 100
                    color: "#868686"
                    radius: 5
                    opacity: 0
                    border.width: 2
                }
            }
        }
    }
}

and Main.qml

import QtQuick 2.6
import QtQuick.Window 2.2

Window {
    visible: true
    width: 500
    height: 400
    title: qsTr("Hello World")
    property int rectIdx

    MainForm {
        anchors.fill: parent
        mouseArea.onClicked: {
            //console.log(qsTr('Clicked on background. Text: "' + textEdit.text + '"'))
        }
        mouseArea.onPressed: {
            rectIdx = getRectIdx(mouseArea.mouseX, mouseArea.mouseY)
            //console.log(qsTr('Rect Coord "' + xCord + ', ' + yCord + '" idx: ' + rectIdx))
            repeater.itemAt(rectIdx).opacity = .8
        }
        mouseArea.onReleased: {
            rectIdx = getRectIdx(mouseArea.mouseX, mouseArea.mouseY)
            //console.log(qsTr('Rect Coord "' + xCord + ', ' + yCord + '" idx: ' + rectIdx))
            repeater.itemAt(rectIdx).opacity = 0
        }
    }
    function getRectIdx(x, y){
        x = x/100;
        y = y/100;
        y = y|0;
        x = x|0;
        return y * 5 + x;
    }
}

alt text

SGaist

qrc files are read-only and end up as a .cpp file that is compiled.

Your images are not found likely because you're using a relative path.

cdw3423

@SGaist
I figured out the relative path problem. But I can't find anyway to get an image added as built in resource using Qt Creator 4.2.1 with out manually editing qml.qrc. I have to reload the project after that edit I think. If there is some way to add items to the resources using the Qt Creator UI, I can't find it.

jsulm

@cdw3423

Select resource file in QtCreator
Right click on it then "Open With/Resource Editor"
Add/remove files there

cdw3423

@jsulm Last night when I read your post, I was sure I had tried to right click and nothing happened. But I tried it and it did work. Today I decided to do some more experimenting and once again right clicking on the resource file doesn't do anything. So I opened a few different projects and still no joy. I'm not sure if this is a bug or if somehow I am in a mode that won't let me see the right click options. I tried it with and with out a project running.

SGaist

What happens if you just select the .qrc file ? You should get a nice editor.

cdw3423

@SGaist
Last night when I tried, it was all working, but not today. Maybe I will install it on a different computer and see what happens.