Adding GUI Tests to CI without display

Wuzi

Hello,

I have some GUI CI Tests in the github CI Gittyup. We have an ubuntu system with wayland.

Probably because of this the GUI tests are failing because xcb cannot be loaded correctly

This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, xcb.

I tried to setup Qt_QPA_PLATFORM to wayland but there I get the error message:

error: XDG_RUNTIME_DIR not set in the environment.
Failed to create wl_display (No such file or directory)
error: XDG_RUNTIME_DIR not set in the environment.
Failed to create wl_display (No such file or directory)
qt.qpa.plugin: Could not load the Qt platform plugin "wayland" in "/home/runner/work/Gittyup/Qt/5.15.2/gcc_64/plugins/platforms" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Which platform do I have to use to get the CI running also GUI tests?

SGaist

Hi,

@Wuzi said in Adding GUI Tests to CI without display:

offscreen

Wuzi

Hi @SGaist, thank you for your reply. I tried offscreen, but then the test does not run correctly and the CI do never ends (offscreen)

SGaist

You can run your tests using that plugin on your machine to check what is going on.

Wuzi

Yes I tried and there the test runs without any problems

KH-219Design

Based on the existing comments in this thread, it appears that using the offscreen plugin does allow the GUI to launch during CI, but then the GUI tests do not behave as expected.

On the one hand, that would leave room for the possibility that the tests are simply flaky tests, and something about the test logic needs fixing.

However, in case you want to try another approach to remove the offscreen plugin from the equation, then I will describe my approach here.

I am able to launch my Qt/QML GUI application in CI, and I was unaware of any offscreen plugin.

What I do is to launch the GUI "just like I normally would" (normally being that I would assume a graphics display is available and a graphics server is running). The crucial added "trick" is to use Xvfb.

All of my scripts (and the CI logs) are available for you to inspect. Start with: run_all_tests.sh ... and look at these parts:

Starting Xvfb

if [[ -n ${XDISPLAY-} ]]; then
  if [[ "$OSTYPE" != "darwin"* ]]; then
    # 'coproc' so that SIGINT will still work (https://unix.stackexchange.com/a/478697/11592)
    coproc Xvfb ${XDISPLAY} -screen 0 1024x768x16
    VIRT_FB_PID=$!
    sleep 4 # time to (probabilistically) ensure that Xvfb has started
  fi
fi

Launching the GUI app

# run gui tests which execute the actual app binary:
tools/gui_test/launch_gui_for_display.sh "${XDISPLAY}"

Shutting down Xvfb (so CI can end cleanly without any system error codes)

if [[ -n ${XDISPLAY-} ]]; then
  if [[ "$OSTYPE" != "darwin"* ]]; then
    kill -SIGINT $VIRT_FB_PID
  fi
fi

The CI configurations that I use run on both:

GitHub CI, and
BitBucket CI

Wuzi

Hi @KH-219Design, but on Ubunutu-latest we have Wayland and not X is that then still possible?

When I tried to use xcb it fails. When checking then the plugin I get (So some x libraries are missing):
ldd $Qt5_DIR/plugins/platforms/libqxcb.so returns:


linux-vdso.so.1 (0x00007ffc127e7000)
	libQt5XcbQpa.so.5 => /home/runner/work/Gittyup/Qt/5.15.2/gcc_64/lib/libQt5XcbQpa.so.5 (0x00007fa87bcf4000)
	libfontconfig.so.1 => /lib/x86_64-linux-gnu/libfontconfig.so.1 (0x00007fa87bc9d000)
	libfreetype.so.6 => /lib/x86_64-linux-gnu/libfreetype.so.6 (0x00007fa87bbde000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fa87bbc2000)
	libQt5Gui.so.5 => /home/runner/work/Gittyup/Qt/5.15.2/gcc_64/lib/libQt5Gui.so.5 (0x00007fa87b291000)
	libQt5DBus.so.5 => /home/runner/work/Gittyup/Qt/5.15.2/gcc_64/lib/libQt5DBus.so.5 (0x00007fa87b003000)
	libQt5Core.so.5 => /home/runner/work/Gittyup/Qt/5.15.2/gcc_64/lib/libQt5Core.so.5 (0x00007fa87a80d000)
	libGL.so.1 => /lib/x86_64-linux-gnu/libGL.so.1 (0x00007fa87a785000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fa87a762000)
	libX11-xcb.so.1 => /lib/x86_64-linux-gnu/libX11-xcb.so.1 (0x00007fa87a75d000)
	libxcb-icccm.so.4 => not found
	libxcb-image.so.0 => not found
	libxcb-shm.so.0 => /lib/x86_64-linux-gnu/libxcb-shm.so.0 (0x00007fa87a756000)
	libxcb-util.so.1 => not found
	libxcb-keysyms.so.1 => not found
	libxcb-randr.so.0 => not found
	libxcb-render-util.so.0 => not found
	libxcb-render.so.0 => /lib/x86_64-linux-gnu/libxcb-render.so.0 (0x00007fa87a745000)
	libxcb-shape.so.0 => /lib/x86_64-linux-gnu/libxcb-shape.so.0 (0x00007fa87a740000)
	libxcb-sync.so.1 => /lib/x86_64-linux-gnu/libxcb-sync.so.1 (0x00007fa87a736000)
	libxcb-xfixes.so.0 => /lib/x86_64-linux-gnu/libxcb-xfixes.so.0 (0x00007fa87a72c000)
	libxcb-xinerama.so.0 => not found
	libxcb-xkb.so.1 => /lib/x86_64-linux-gnu/libxcb-xkb.so.1 (0x00007fa87a70e000)
	libxcb.so.1 => /lib/x86_64-linux-gnu/libxcb.so.1 (0x00007fa87a6e2000)
	libXext.so.6 => /lib/x86_64-linux-gnu/libXext.so.6 (0x00007fa87a6cd000)
	libX11.so.6 => /lib/x86_64-linux-gnu/libX11.so.6 (0x00007fa87a590000)
	libxkbcommon-x11.so.0 => /lib/x86_64-linux-gnu/libxkbcommon-x11.so.0 (0x00007fa87a585000)
	libxkbcommon.so.0 => /lib/x86_64-linux-gnu/libxkbcommon.so.0 (0x00007fa87a543000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fa87a53d000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fa87a359000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fa87a20a000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fa87a1ef000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa879ffd000)
	libxcb-icccm.so.4 => not found
	libxcb-image.so.0 => not found
	libxcb-util.so.1 => not found
	libxcb-keysyms.so.1 => not found
	libxcb-randr.so.0 => not found
	libxcb-render-util.so.0 => not found
	libxcb-xinerama.so.0 => not found
	libgthread-2.0.so.0 => /lib/x86_64-linux-gnu/libgthread-2.0.so.0 (0x00007fa879ff4000)
	libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fa879ecb000)
	libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fa879e9d000)
	libuuid.so.1 => /lib/x86_64-linux-gnu/libuuid.so.1 (0x00007fa879e94000)
	libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007fa879e5c000)
	libdbus-1.so.3 => /lib/x86_64-linux-gnu/libdbus-1.so.3 (0x00007fa879e09000)
	libicui18n.so.56 => /home/runner/work/Gittyup/Qt/5.15.2/gcc_64/lib/libicui18n.so.56 (0x00007fa879970000)
	libicuuc.so.56 => /home/runner/work/Gittyup/Qt/5.15.2/gcc_64/lib/libicuuc.so.56 (0x00007fa8795b8000)
	libicudata.so.56 => /home/runner/work/Gittyup/Qt/5.15.2/gcc_64/lib/libicudata.so.56 (0x00007fa877bd5000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa87c253000)
	libGLdispatch.so.0 => /lib/x86_64-linux-gnu/libGLdispatch.so.0 (0x00007fa877b1d000)
	libGLX.so.0 => /lib/x86_64-linux-gnu/libGLX.so.0 (0x00007fa877ae7000)
	libXau.so.6 => /lib/x86_64-linux-gnu/libXau.so.6 (0x00007fa877ae1000)
	libXdmcp.so.6 => /lib/x86_64-linux-gnu/libXdmcp.so.6 (0x00007fa877ad9000)
	libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fa877a66000)
	libsystemd.so.0 => /lib/x86_64-linux-gnu/libsystemd.so.0 (0x00007fa8779b7000)
	libbsd.so.0 => /lib/x86_64-linux-gnu/libbsd.so.0 (0x00007fa87799b000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fa877990000)
	liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fa877967000)
	liblz4.so.1 => /lib/x86_64-linux-gnu/liblz4.so.1 (0x00007fa877946000)
	libgcrypt.so.20 => /lib/x86_64-linux-gnu/libgcrypt.so.20 (0x00007fa877828000)
	libgpg-error.so.0 => /lib/x86_64-linux-gnu/libgpg-error.so.0 (0x00007fa877803000)

KH-219Design

@Wuzi my apologies for skipping over your mention of Wayland. Silly oversight on my part.

I do not have much experience with Wayland.

My first question would be whether you are comfortable making an assumption that if the Qt app runs as intended on X11, then it will (with high likelihood) run as intended on Wayland. If so, then you should be OK with testing on X11 (and Xvfb) but running/deploying on a Wayland desktop.

If it is a requirement that you test under Wayland, then further research is needed. I found a few links, but I cannot vouch for them:

The missing libxcb libs should be easy to remedy:

sudo apt-get --assume-yes install \
  libxcb-icccm4 \
  libxcb-image0 \
  libxcb-keysyms1 \
  libxcb-randr0 \
  libxcb-render-util0 \
  libxcb-render0 \
  libxcb-xinerama0 \
  libxcb-xkb1 \
  libxkbcommon-x11-0 \
  libxkbcommon0

Wuzi

Hi @KH-219Design ,

installing the missing libraries worked. Now I get only the error message: "could not conenct to display". Due to the unavailability of the screen this is clear.
echo $DISPLAY returns nothing
Found this issue on Github: https://github.com/therecipe/qt/issues/775
Tried setting to minimal did not work

KH-219Design

What you describe sounds like expected behavior until you get Xvfb running properly.

Since you do appear to be getting the results I expect (results which match my experience), I think it would be fruitful for you to go to the bash scripts I linked to in my repository and learn/borrow some more content from there. (I don't want to reproduce all of it here in this thread.)

I have this working in GitHub CI and Bitbucket CI.

In a headless CI environment, it is expected to not have any DISPLAY string/variable until you explicitly make it happen with Xvfb.

(As you will see when reading my CI scripts), when I launch Xvfb I tell it what display variable (string) I want it to use:

coproc Xvfb ${XDISPLAY} -screen 0 1024x768x16

Then later (in launch_gui_for_display.sh), I make sure that the environment "knows" that DISPLAY contains that same string that Xvfb was instructed to use:

if [[ "$OSTYPE" != "darwin"* ]]; then
  echo "reading single argument (should be either empty or ':1' to choose display)"
  export DISPLAY="${1:-"$DISPLAY"}"
fi

Virgo

It's too late and probably against the rules, but whatever

I think the issue is the environment name QT_QPA_PLATFORM should be all caps. Your t is in lower case, it won't work.