Test under xvfb fail after upgrade to qt 5.15.0 for python

[Czaki]

I have my Python application which use qt for GUI. I use qtpy as wrapper so my application can use PyQt5 or PySide2 as frontend. And for both backends for version 5.15.0 when tests are executed on CI server it fails with qt widgets test with sigabort (Test for Windows and MacOS pass) .

Test pass on local machine.

What can change in qt that it causes such error? It happens on Github Actions and Azure Pipelines on ubuntu image.

Link to project:

https://github.com/4DNucleome/PartSeg/tree/develop

Sample fail: https://github.com/4DNucleome/PartSeg/runs/726680128?check_suite_focus=true

[Czaki]

My mistake. I use it as frontend, not backend. This fail on test custom widgets

I try to create smaller example.

[Czaki]

Here is example https://github.com/Czaki/sample_qt_error
log here https://github.com/Czaki/sample_qt_error/runs/727566764?check_suite_focus=true

[Czaki]

How this respond on my question about changes in Qt 5.15?. My code works with Qt 5.14. It fail on QApplication constructor call.

here is log for both bind (PyQt5 and PySide2) for Qt in version 5.14 and 5.15.

https://github.com/Czaki/sample_qt_error/runs/727854634?check_suite_focus=true

For Qt 5.14 all test pass. For Qt 5.15 it fail on QApplication creation.

On My PC it pass for both version. On server without X (but with Xvfb) it fails.

[KH-219Design]

@Czaki I am only just now arriving on this thread, and when I look at the GitHub actions now (for PartSeg) I see a lot of "green" (successful) jobs. Did you fix the problem?

Was this your fix? https://github.com/4DNucleome/PartSeg/commit/d7771b213007e9e44ef4f1e159c5258cbb2dbadf

(I don't use python, but I, too, have run into weird issues with the Azure/Microsoft configuration of the GitHub Ubuntu runner environment.)

[Czaki]

@KH-219Design I look like it point to wrong build, but I'm not sure how it can change. Current build are green because I block 5.15.0 release
Maybe this link will work: https://github.com/4DNucleome/PartSeg/runs/726685612?check_suite_focus=true

In this repository I reproduce this error https://github.com/Czaki/sample_qt_error
It shows that this is problem with QApplication in this line

        _qapp_instance = qt_api.QApplication(qapp_args)

where qapp_args is empty list

@Denni-0 This code is test runner (pytest). Main application is run in proper way https://github.com/4DNucleome/PartSeg/blob/develop/package/PartSeg/launcher_main.py

And I can run code on linux with running x11 server. Problem happens on server with x emulation with xvfb. But CI provider does not provide server with running x11, so xvfb and Xdummy are only options to test gui components. But I do not know how to setup Xdummy.

But also test runner create QApplication object before test start.

[Czaki]

In other place I found that my problems comes from this change:
https://codereview.qt-project.org/c/qt/qtbase/+/253905

so I need to install xcb libraries with:

sudo apt-get install -y libdbus-1-3 libxkbcommon-x11-0 libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-randr0 libxcb-render-util0 libxcb-xinerama0 libxcb-xfixes0

(not sure if all needed)

I use simple file:

from qtpy.QtWidgets import QApplication, QMainWindow

app = QApplication([])
window = QMainWindow()
window.show()
app.exec_()

with defined QT_DEBUG_PLUGINS=1 in environment to get information which libraries are missing

@Denni-0 This code starts one of 3 applications base on command line arguments. and contains function _test_imports which test if all libraries are properly bundled when freezing with pyinstaller.

I'm not sure If you can simplify it. qtpy is simple wrapper which detect qt libraries and allow to control it if have multiple installed.

Base on PEP-8 all imports of python standard library should be before import of third parts library.

[Czaki]

@Denni-0 Could you point any part of standard python library which may breaks Qt?

I know that there are some third part library (like matplotlib, vispy) which may break when using Qt with wrong import order. But I do not see any option when part of python standard library may break Qt.

[JonB]

@Czaki said in Test under xvfb fail after upgrade to qt 5.15.0 for python:

@Denni-0 Could you point any part of standard python library which may breaks Qt?

Excellent question.

[KH-219Design]

@Czaki I'm laughing (at myself!) as I write this, because I'm about to say almost the same thing I said yesterday.

I'm not crazy! And I have read the latest messages.

Seeing that this was this still unsolved this morning, and given that I still have a lot of personal curiosity about things that go wrong in GitHub CI specifically, I came here thinking "well the post did link to a minimum repro of the issue."

So I decided I would dig in and fork the "Czaki/sample_qt_error" repository and see if I could diagnose it.

But when I go to https://github.com/Czaki/sample_qt_error/actions/runs/121851186 it looks like there is a "green" (successful) job. So was this the fix? https://github.com/Czaki/sample_qt_error/commit/758bc7b3420330529cc1e713f511d61ae665d62b

If so, congrats on narrowing it down, and thank you for sharing this with the forum. (And sorry we did not provide the answer!)

If I am once again misinterpreting a "green" job, then today I will actually collaborate if this is still an ongoing problem.

[KH-219Design]

In this case, there seems to be ample evidence that simply upgrading to Py Qt 5.15.0 is the change that broke the CI jobs.

However, I thought I should share one "trick" I have been using on GitHub CI, which has helped me identify cases where changes made by GitHub have been the sudden cause of broken CI jobs.

In my CI job script: https://github.com/219-design/qt-qml-project-template-with-ci/blob/1b981d218c7/run_all_tests.sh

I print into the CI log the following information before I start my compilation and test commands:

  # Try various ways to print OS version info.
  # This lets us keep a record of this in our CI logs,
  # in case the CI docker images change.
  uname -a       || true
  lsb_release -a || true
  gcc --version  || true  # oddly, gcc often prints great OS information
  cat /etc/issue || true

  # What environment variables did the C.I. system set? Print them:
  env

If a job suddenly fails one day, then I download the GitHub logs from the most recent SUCCESS and from the failure jobs, and I do a diff of the log. This can reveal changes in the GitHub runner environment.

GitHub also prints similar info at the top of each job's log. Their built-in descriptive logging looks like:

2020-06-02T01:29:00.2554826Z ##[section]Finishing: Request a runner to run this job
2020-06-02T01:29:04.9619878Z Current runner version: '2.263.0'
2020-06-02T01:29:04.9642595Z ##[group]Operating System
2020-06-02T01:29:04.9643252Z Ubuntu
2020-06-02T01:29:04.9643359Z 18.04.4
2020-06-02T01:29:04.9643510Z LTS
2020-06-02T01:29:04.9643685Z ##[endgroup]
2020-06-02T01:29:04.9643850Z ##[group]Virtual Environment
2020-06-02T01:29:04.9643978Z Environment: ubuntu-18.04
2020-06-02T01:29:04.9644127Z Version: 20200525.2
2020-06-02T01:29:04.9644336Z Included Software: https://github.com/actions/virtual-environments/blob/ubuntu18/20200525.2/images/linux/Ubuntu1804-README.md

So when jobs fail mysteriously, I have tended to "diff the logs" specifically to check that no GitHub changes happened.

Normally I adhere to "the first rule of bug hunting is assume the bug is your own." But GitHub Actions as a feature is in its infancy still, and they have made many rapid changes and I have seen it break several times. (see: https://github.com/219-design/qt-qml-project-template-with-ci/commit/7eda342bb693738d6c9d912619a34cacb498b7c1 and https://github.com/219-design/qt-qml-project-template-with-ci/pull/12)

I offer this all as "future advice". Again, I see that the change to Qt 5.15.0 is sufficient to explain the broken jobs for @Czaki .

[Czaki]

@KH-219Design I fix it and describe here: https://forum.qt.io/post/598392 (few post above).

And I was sure that problem is in changes in Qt, because exactly same code works for QT 5.14.2 and fail for Qt 5.15.0 at same time, at same machine. (one Github Actions job)

@Denni-0 said in Test under xvfb fail after upgrade to qt 5.15.0 for python:

Now of course if you would care to dig that deeply into Qt to understand why that occurs and specifically with what it occurs with, as THEY (Qt) have describe it does. Then by all means do so, and share your findings. I am personally fine with adhering to their (Qt's) advice of declaring the Qt stuff first (I guess they do not adhere to PEP-8 either imagine that) followed by the python stuff followed by anything else.

Could you provide link to article about this? Because I open Pyside2/__init__.py and it starts from

from __future__ import print_function
import os
import sys

Which is import of Python standard library

If you import sys and check len(sys.modules.keys()) you will got information that above 60 modules are already loaded (Python 3.6.8 on Linux gives 63 modules loade). So import of any of this module cannot break Qt.

Next Python-Qt is not pure Python code

I see no difference for Qt and other libraries. Many of common used libraries has extensions in C ore are simply wrapper for C/C++ library.

This putting all python standard library elements before third party libraries id good idea to see dependences of code. all packages from second section need to be installed using package manager.

I understand how compilers works. I even write simple one. You produce wall of text which is not connected with my issue without any arguments. Breaking PEP-8 block usage of many tools which do code analysis, which speed-up and increase of quality of code developing.

Finally it shown that whole problem is connected with drop single library from Qt build.

However to prove this the code would have to be compiled without using qtpy and using perhaps that suggestion I made. If it were I and I was developing this project -- I would have 2 branches one that uses qtpy and one that does not -- and when compiling and running I would always do both in order to determine if qtpy (that extra layer that got added that I have no control of) is the issue or not.

I do qtpy code review and it is enough.

And I test my code against two version of Qt at same time and one pass and other fail. With exactly same version of other packages (qtpy also) so this need to be connected with Qt change.

I also create short code example which in one version import from PySide2 not qtpy and it also fail.

[KH-219Design]

@Czaki Ah. I actually clicked through to https://codereview.qt-project.org/c/qt/qtbase/+/253905 this time. Fascinating.

Excellent detective work.

[Czaki]

My main backend is PyQt5 but it is from another company (Riverbank Computing). But, because of licensing, some people prefer PySide2 over PyQt5. So I try to write universal code. But it happens that some release of PySide2 breaks, when PyQt5 works. But fortunately next releases of PySide2 fixes it.

I start this thread because It fail on both Qt packages, which means that it is connected with Qt code, not python part.

And I show examples based on PySide2 because this package is produced by Qt company.