QWS VNC doesn't work on Portrait mode



  • Hi,

    I am working on embedded product which has 7" LCD display. And the core is TI AM335x.

    Since the display by default is, Landscape mode and I need to rotate the screen to portrait mode as per the requirements.

    Since I need screen rotation, I use a QWS option "Transformed:Rot270" to rotate the screen. And it works cool.

    Now I would like to extend the display to my linux as well. So, I use VNC for that.
    When I tested VNC, it worked excellent in case of TI AM335x EVM board.
    The reason is that, I have never rotated the screen in the EVM board.

    Now, in my application, I need to rotate the screen.

    The export option I have set is,
    export QWS_DISPLAY="Multi: VNC:size=480x640:0 Transformed:Rot270:LinuxFb:size=480x640:1 Transformed:Rot270 :2"

    With this In the desktop, I could see the VNC viewer screen was rotated and displayed as I wanted. And also, I have used the mouse to operate the UI and it was working perfect.

    But in the LCD UI, I could see two UI's of actual UI. Meaning, One in the landscape mode and other one in the portrait mode.
    And I can actually operate the one which is in portrait mode. The landscape mode UI was actually from VNC (may be).

    Then later, I was trying various combinations for QWS, but could not corner the problem.

    Then I googled to see if there are fixes. That time, I came across this:
    https://bugreports.qt.io/browse/QTBUG-12290
    It looks like it is a bug from Qt related to screen rotation.

    I wanted to apply this patch and see if it solves my problem.

    After applying the patch, my observations are

    • Both LCD and Linux shows the UI exactly same.
    • When I operate the UI from Linux, it works perfect.
    • When I operate the UI from LCD, it doesnt work. It looks like the touch inputs are still without screen rotate.
      Visualize the screen in portrait mode (rotated screen). Meaning, assume there is a button in top left corner. If I click the button from Linux VNC, yes it works. When I touch the same position - top left in the LCD UI, there is no response. But, if I touch the bottom left, button responds. It seems like the touch inputs are not rotated.

    I tried my best to explain the problem. Please let me know if you have any thoughts on this.

    Thank you,
    Kumara


  • Lifetime Qt Champion

    Hi,

    I'd look at the sendMouseEvent patch part, it's probably there that the dispatching is not done correctly



  • @SGaist Hi Sam, Thank you for taking time and looking at the implementation.

    How to go about?

    Thanks,
    Kumara



  • @SGaist This was the else part added as part of the patch (that is attached in Qt bug)

        if (qt_screen->isTransformed()) 
        {
            QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
            tpos = qt_screen->mapFromDevice(pos, s);
        } 
        else 
        {
            //tpos = pos;
            // Only in case of Multi Display
            // If main screen is not transformed, then check if sub screens are transformed
            QList<QScreen*> screens = qt_screen->subScreens();
            if (screens.size() && !bRemote) 
            {
                for (int i = 0; i < screens.size(); ++i) 
                {
                    QScreen *screen = screens.at(i);
                    if (screen->isTransformed())
                    {
                        QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                        tpos = screen->mapFromDevice(pos, s);
                        break;
                    }
                }
            }
        }


  • @SGaist This Qt bug is not fixed in any of Qt 4.8 versions?


  • Lifetime Qt Champion

    It's still the same in 4.8.7



  • @SGaist Ok Sam. Do you see any way to fix this problem?

    Thank you.


  • Lifetime Qt Champion

    If I understand the patch correctly, it only check other screens if the main screen is not transformed. Since you have more that one screen transformed, you should check all screens (main + sub-screens)



  • @SGaist Then it means that, the else part of code, should be done for if condition as well?

        if (qt_screen->isTransformed()) 
        {
            QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
            tpos = qt_screen->mapFromDevice(pos, s);
        } 
        //tpos = pos;
        // Only in case of Multi Display
        // If main screen is not transformed, then check if sub screens are transformed
        QList<QScreen*> screens = qt_screen->subScreens();
        if (screens.size() && !bRemote) 
        {
            for (int i = 0; i < screens.size(); ++i) 
            {
                QScreen *screen = screens.at(i);
                if (screen->isTransformed())
                {
                    QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                    tpos = screen->mapFromDevice(pos, s);
                    break;
                }
            }
        }
    

    Is it like that?


  • Lifetime Qt Champion

    I'd remove also the bRemote, just check them all



  • @SGaist Hmm, let me test this.

    Just for better understanding attaching the modified code:

    if (qt_screen->isTransformed()) 
        {
            QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
            tpos = qt_screen->mapFromDevice(pos, s);
        } 
        //tpos = pos;
        // Only in case of Multi Display
        // If main screen is not transformed, then check if sub screens are transformed
        QList<QScreen*> screens = qt_screen->subScreens();
        if (screens.size()) 
        {
            for (int i = 0; i < screens.size(); ++i) 
            {
                QScreen *screen = screens.at(i);
                if (screen->isTransformed())
                {
                    QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                    tpos = screen->mapFromDevice(pos, s);
                    break;
                }
            }
        } 
    

    I'll test and keep you posted..



  • No success yet!

    While running the application with Multi for VNC, still the touch inputs on the device is still not rotated.


  • Lifetime Qt Champion

    Silly idea (and just to verify) but what about just dropping the if and do the transform for all screens anyway ?



  • Just for more information:

    export QWS_DISPLAY="Multi: VNC:0:size=480x640 Transformed:rot0"
    

    This displays the UI in a landscape mode and the touch input works as per the UI.

    export QWS_DISPLAY="Multi: VNC:0:size=480x640 Transformed:rot90"
    

    This rotates the UI to 90 degrees. But the touch input is as same as 0 degrees.

    This clearly indicates that the display transformation works good but the transformation to touch input mapping is not working.



  • @SGaist Sam, can you please send me the code snippet to be on same page?

    Thanks,
    Kumara



  • @SGaist You mean to try this kind of code:

    QList<QScreen*> screens = qt_screen->subScreens();
    if (screens.size()) 
    {
        for (int i = 0; i < screens.size(); ++i) 
        {
            QScreen *screen = screens.at(i);
            QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
            tpos = screen->mapFromDevice(pos, s);
            break;
        }
    }
    

    instead of

    if (qt_screen->isTransformed()) 
        {
            QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
            tpos = qt_screen->mapFromDevice(pos, s);
        } 
        else 
        {
            //tpos = pos;
            // Only in case of Multi Display
            // If main screen is not transformed, then check if sub screens are transformed
            QList<QScreen*> screens = qt_screen->subScreens();
            if (screens.size() && !bRemote) 
            {
                for (int i = 0; i < screens.size(); ++i) 
                {
                    QScreen *screen = screens.at(i);
                    if (screen->isTransformed())
                    {
                        QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                        tpos = screen->mapFromDevice(pos, s);
                        break;
                    }
                }
            }
        }
    

    Sam, if you confirm, I can make these changes and build my packages.

    Thanks,
    Kumara


  • Lifetime Qt Champion

    Yes that's what I had in mind, without forgetting the main screen first.



  • @SGaist Oh ya, I missed the main screen

    So here is the code which I can do the test:

    // Main screen
    QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
    tpos = qt_screen->mapFromDevice(pos, s);
    
    // Subscreens
    QList<QScreen*> screens = qt_screen->subScreens();
    if (screens.size()) 
    {
        for (int i = 0; i < screens.size(); ++i) 
        {
            QScreen *screen = screens.at(i);
            QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
            tpos = screen->mapFromDevice(pos, s);
            break;
        }
    }
    

    Right?



  • @SGaist Below the patch code, I see

        QPoint tpos;
        // Main screen
        QSize s = QSize(qt_screen->deviceWidth(), qt_screen->deviceHeight());
        tpos = qt_screen->mapFromDevice(pos, s);
    
        // Sub screens
        QList<QScreen*> screens = qt_screen->subScreens();
        if (screens.size()) 
        {
            for (int i = 0; i < screens.size(); ++i) 
            {
                QScreen *screen = screens.at(i);
                QSize s = QSize(screen->deviceWidth(), screen->deviceHeight());
                tpos = screen->mapFromDevice(pos, s);
                break;
            }
        }
    
        if (qt_last_x) {
             *qt_last_x = tpos.x();
             *qt_last_y = tpos.y();
        }
        QWSServer::mousePosition = tpos;
        qwsServerPrivate->mouseState = state;
    

    We use the tpos and mouse the mousePosition just once. Even though, we have many subscreens, we just do it once.

    Is this correct?

    Should we not do this in a loop for all the screens?

    Please let me know.



  • Looks like I am in a wrong direction.

    I suspect "Transformed" driver is not having problem.

    I think, it is "multi" screen driver is what not working good.

    Because, when VNC option is selected, Transformed works food.
    When LinuxFb is selected, Transformed works good.

    When both are used, there lies the problem.

    I guess, we may have to look at Multi Screen driver source code to see how it has been implemented.

    Can you please point me to the right location?

    Thanks,
    Kumara


  • Lifetime Qt Champion

    I'd take a look a the qscreenmulti_qws.cpp file



  • @SGaist Oh!
    Any thing you found?



  • @SGaist Later I gave a try with

    export QWS_DISPLAY="VNC:Transformed:Rot270"
    

    I dont know the magic behind this.

    In my LCD display UI, looks good. And even touch works well.

    And now, in the VNC client, Application was rotated. So, I used RealVNC client and rotated the screen to 270.
    With that, UI looks good and even operation works well.

    I dont know what magic behind the scene.

    But the problem I see is, display refresh in VNC client. The screen doesnt gets refreshed. There is a hug lag while updating the screen..

    I am totally confused now, to understand the problem.

    Based on your experience,
    Where do you see the problem?

    Thanks a lot for your immense support!


  • Lifetime Qt Champion

    90 and 270 are "complementary" meaning that 270 is equivalent to -90 so it might be that the position calculation were done wrong (or just not taking the angle in account correctly)

    Do you mean that with 90, the refreshing is done correctly on both but not the mouse handling and with 270, the refreshing is wrong but the mouse move are handled correctly ?



  • @SGaist No. With both 90 or 270, I can see the mouse move is perfect. But the refreshing is not working good.



  • @SGaist Sam, any further thoughts?

    I don't want to give up. I want to fix this Qt bug. For sure, I will need your / Qt experts help.

    Thank you,
    Kumara



  • Still no success so far.

    Any thoughts on how to handle this?

    Thank you!
    Kumara


  • Lifetime Qt Champion

    Can you remind me the current "multiscreen" setup you have ?



  • This is a quite old topic but what is the current situation for this problem?


Log in to reply
 

Looks like your connection to Qt Forum was lost, please wait while we try to reconnect.