How to change qvncserver pixel format from client

majorRonk

How i can change the pixel format of the qvncserver by sending request from client?
My client tries to sent a new pixel format (f.e. with depth 24 or 16) request like in the following example,
but the server always keeps the default RGB32 format.
My vnc server is a simple qml VncItem item.
Unfortunately VncItem does not have much functionality like changing the format or at least show notifications.
Guess i have to implement my own QVncServer for this.

https://github.com/sandsmark/revncable/blob/0c0aa4188c119cc26cc31c067c97526bf6101b0a/rfbclientcls.cpp#L874

bool sendClientSetPixelFormat(pixelFormatStruct pixel)
{
    unsigned char vncClientData[20];

    vncClientData[0] = 0; //message type
    vncClientData[1] = 0x00;
    vncClientData[2] = 0x00;
    vncClientData[3] = 0x00;
    vncClientData[4] = pixel.bpp;
    vncClientData[5] = pixel.depth;
    vncClientData[6] = pixel.big_endian_flag;
    vncClientData[7] = pixel.true_color_flag;
    vncClientData[8] = (pixel.red_max >> 8) & 0xFF;
    vncClientData[9] = (pixel.red_max >> 0) & 0xFF;
    vncClientData[10] = (pixel.green_max >> 8) & 0xFF;
    vncClientData[11] = (pixel.green_max >> 0) & 0xFF;
    vncClientData[12] = (pixel.blue_max >> 8) & 0xFF;
    vncClientData[13] = (pixel.blue_max >> 0) & 0xFF;
    vncClientData[14] = pixel.red_shift;
    vncClientData[15] = pixel.green_shift;
    vncClientData[16] = pixel.blue_shift;
    vncClientData[17] = 0x00;
    vncClientData[18] = 0x00;
    vncClientData[19] = 0x00;

    if (this->writeToHost(vncClientData,20) != 20)
    {
        qDebug("   fail to set pixel format");
        return false;
    }

    return true;
}

SGaist

Hi,

Based on the QVncServer constructor, I would say it seems that it's not something that can be changed at run time. Especially since there are no setters for the image format.

majorRonk

I created derived class from QVncServer and call provideImage after signal requestImage was triggered.
I pass the image format in constructor. It works for RGB32 but for any other format QVncServer crashes in provideImage and i cannot see the location where it fails.
output for RGB888 is "qt.vncserver: QVNCServer: don't support 24bpp display"
RGB16 and Indexed8 crashes after "qt.vncserver: Making new map"

jsulm

@majorRonk said in How to change qvncserver pixel format from client:

i cannot see the location where it fails

Run in debugger and check the stack trace after the crash

SGaist

I just saw that QVncServer is part of the commercial offering. I would recommend checking with the Qt Company as that module's code does not seem to be available.

majorRonk

@jsulm
It shows only Disassembler.

SGaist

@majorRonk are you sure you are running a debug build ?

majorRonk

I have misinterpreted the "screenFormat" of QVncServer. Its the rendered format, not the format the server sends to the client.
Started TigerVNCViewer with different color depths and it seems to work as expected.
After i used the correct bpp, redshift etc. values for different pixel formats my custom qt vnc client is also working again.

The only problem which is currently left is:
How to switch back from a lower bpp to higher bpp pixel format at runtime?
Everything else is working fine but when i switch back from 16 to 24 or 32 bpp,
my client / server communication gets out of sync, reading strange values.
But thats more like a generral rfb question, how to sync client/server again f.e. with something like a hard reset.

SGaist

You should take a look at the vnc platform plugin implementation. I remember seeing there something related to adjustment.