Strange QSettings issue with Qt6 on Jenkins

SGaist

Hi,

In addition to the questions of my fellows, one thing to consider for the tests is to write the file to a known read/write path rather than relying on system defaults.

kehr_I

Hm, the forum seems to have eaten my post. So let's repeat it.

We always have to keep in mind, that the test does not fail on our local machines - or any virtual machine if executed manually. It only fails if executed by a Jenkins pipeline.
That gives rise to my assumption, that the issue is not really with the test.

File:
The QSettings format is ini-format.
I tried several variants, explicitly setting the directory for the ini-file to a location, which was used to write additional test data (verified). Didn't change the outcome.
I added assertions verifying the existence of the ini-file before the access through the object under test. Didn't change the outcome.
I even connected to the VM and had a look at the ini-file. It looked reasonable. That's why I assume, that it is the reading, which fails, not the writing.

Actions:
Executed the tests

on local machine (win11): pass Qt5-version, pass Qt6-version
on Jenkins VM using pipeline (win10/11): pass Qt5-version, FAIL Qt6-version
on Jenkins VM connected, executing manually (win10/11): pass Qt5-version, pass Qt6-version

Libraries:
We are using different versions of Qt libraries, all built by ourselves, so not pre-built versions:
a) Qt 5.15.2: built manually using the qt-everywhere package, then wrapped into a binary conan1 package
b) Qt 6.7.3: built using conan1 from conancenter
c) Qt 5.15.16: which is a conan-patched version of 5.15.2, built with conan 2.12 from conancenter
d) Qt 6.8.3: built using conan2 (still a PR on conancenter)

I haven't seen the issue happening with a), at least not on a regular basis. With b) and d) it is always reproducible. With c) I have seen it, when I disable the failing tests for Qt6 (they pass, when the Qt6-version tests are enabled).

When writing that last paragraph... could it be, that I need to add further options to the conan builds?

JonB

@kehr_I

Do you want to show the code the test uses, for both the QSettings creation/access and the check "verifying the existence of the ini-file"? What about verifying the readability by the current user of the file?
"explicitly setting the directory for the ini-file to a location": I presume this means QSettings::setPath()(?). As @SGaist wrote, try an absolute path for QSettings::QSettings(const QString &fileName, QSettings::Format format, QObject *parent = nullptr). What if you put that in some "publicly read/write location", like a C:\Temp, rather than anything to do with the current user?
Switch off QSettings::setFallbacksEnabled(bool b).
"I tried several variants, explicitly setting the directory for the ini-file to a location, which was used to write additional test data (verified). Didn't change the outcome." Do you mean the additional data was written to the same file as intended to read from or to some other file/location? In your test can you then read back this additional data from wherever it was written to?

kehr_I

I can show you a simplified version:
myDialog.hpp:

#pragma once
#include <QDialog>
class MyDialog: public QDialog {
Q_OBJECT
public:
  MyDialog(QWidget* parent = nullptr);
  ~MyDialog() override;
// other members
protected:
  void closeEvent(QCloseEvent* event) override;
  void showEvent(QShowEvent* event) override;
private:
  struct Impl;
  std::unique_ptr<Impl> _p;
};

myDialog.cpp

#include "myDialog.hpp"

#include <QCloseEvent>
#include <QOpenEvent>
// ... Impl declaration and definition ... omitted
MyDialog::MyDialog(QWidget* parent): QDialog{parent}, _p{std::make_unique<Impl>()} {}
MyDialog::~MyDialog() = default;

void MyDialog::closeEvent(QCloseEvent* event)
{
  QSettings settings(QSettings::IniFormat, QSettings::UserScope,
                     QCoreApplication::organizationName(), QCoreApplication::applicationName());
  settings.beginGroup("MyDialog");
  settings.setValue("geometry", saveGeometry());
  settings.endGroup();
  event->accept();
}
void MyDialog::showEvent(QShowEvent* event)
{
  QSettings settings(QSettings::IniFormat, QSettings::UserScope,
                     QCoreApplication::organizationName(), QCoreApplication::applicationName());
  settings.beginGroup("MyDialog");
  restoreGeometry(settings.value("geometry").toByteArray());
  settings.endGroup();
  event->accept();
}

The myDialogTest.cpp:

TEST_F(MyDialogTest, verifyGeometry)
{
  QSettings::setPath(QSettings::IniFormat, QSettings::UserScope, persistedTestSpecificDirectory);
  QCoreApplication::setApplicationName("unit-test"); //QApplication created in test-fixture
  QCoreApplication::setOrganizationName("unit-test");
  QSize sizeNew;
  QPoint positionNew;
  {
    MyDialog widget;
    widget.show();

    auto sizeInitial     = widget.size();
    auto positionInitial = widget.pos();
    sizeNew              = sizeInitial + QSize(5, 5);
    positionNew          = positionInitial + QPoint(-10, 10);
    widget.resize(sizeNew);
    widget.move(positionNew);

    EXPECT_EQ(widget.pos(), positionNew);
    EXPECT_EQ(widget.size(), sizeNew);

    widget.close();
  }
  QSettings s{QSettings::IniFormat, QSettings::UserScope,
                     QCoreApplication::organizationName(), QCoreApplication::applicationName()};
  ASSERT_TRUE(std::filesystem::exists(s.fileName().toStdString()));
  {
    MyDialg widgetNew;
    widgetNew.show();

    EXPECT_EQ(widgetNew.pos().x(), positionNew.x());
    EXPECT_EQ(widgetNew.size().height(), sizeNew.height());
    EXPECT_EQ(widgetNew.pos().y(), positionNew.y());
    EXPECT_EQ(widgetNew.size().width(), sizeNew.width());
  }
}

JonB

@kehr_I
I see nothing about any "path to INI file" in your code.
Consequently I don't even know how you state "The ini-file exists, which is also verified by the unit-test."
As I suggested, why not try using an explicit, absolute path to the file, at least to test behaviour?
If that succeeds but it fails with your shown code then I would suspect your choice of QSettings::UserScope does not lead to the location you expect, perhaps in some particular context of however your "unattended Jenkins pipeline" works.

Also, since you say "versions of Qt libraries, all built by ourselves, so not pre-built versions" you could (temporarily) put in some debug/trace code to find out where it is looking for what file and why it seems to fail on read.

kehr_I

I added the check; I had copied the code from an other branch, than my testing branch for this issue.

With QSettings::setPath(), you should be able to set the path to the exact location, shouldn't you
and verifying with std::filesystem::exists(setting.fileName().toStdString()) should work whether the path had been set or not.
Or am I completely off here?

JonB

@kehr_I
QString QSettings::fileName() const states:

Returns the path where settings written using this QSettings object are stored.

and also references QSettings::isWritable(). It is noticeable to me that it specifically mentions "writing" yet not "reading". Maybe this is significant for your failed-reading case? Plus isWritable() requires a non-readonly file (might yours be read-only, or lack user write permission?). I suggested you test for read/writability.

In principle setPath() ought work, though you don't show it being called or where you set it to. I suggested trying a system-wide path rather than anything to do with, say, the user, at least to test.

However since you have an inexplicable problem I suggested you test with a full, explicit path to QSettings() constructor instead of relying on setPath(), just in case. If it's different we know the cause, if it is not nothing is lost. You could have checked this by now.

kehr_I

@JonB
I connected to the Jenkins virtual machine and had a look at the file, when I had changed the path to the test-directory. The file is there and the content looks reasonable.

The weird thing is, that re-running the test manually, the tests pass; adding an other ctest --rerun-failed to the pipeline doesn't.

And why do the respective tests in the Qt5 branch start to fail when I comment out the same tests in the Qt6 branch?
I've got loads of situations in which QSettings read and write works as expected; only in the Jenkins pipeline it is fussing around - and not for all cases (there is lots of widgets using the same mechanism, only for 2 it fails, on Jenkins).

I'd put my bets on either something's wrong with the configuration of our Jenkins virtual windows machines or there are compiler options missing when I built the conan packages.
Any suggestions which MSVC options these could be?

JonB

@kehr_I
I don't know the answers to any of this, nor do I use Jenkins. But from what you say either your code does not hit the showEvent() code or the path is not what you expect or the content is not correct or there is a problem reading it. You have the opportunity to put in some debug statements around your code to try to show you what is going on. Since you say it is unattended you could send those to some external log file to be examined later. That is what I would do.

Christian Ehrlicher

This is for sure no compiler issue. More an access problem due to a virus scanner or tests running simultaneously.