QSettings file location in Ubuntu ambiguous?

JonB

@Bengt-0 said in QSettings file location in Ubuntu ambiguous?:

Or is XDG_CONFIG_DIRS a Qt related environmental?

No. Qt is using it, but does not set it, it is not a Qt creation. But Qt code might do the picking from the :-list directories or set fileName() not to what you say it should do, I do not know.

https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
https://askubuntu.com/questions/1179729/where-is-xdg-config-dirs-set

$XDG_CONFIG_DIRS defines the preference-ordered set of base directories to search for configuration files in addition to the $XDG_CONFIG_HOME base directory.

XDG is to do with your desktop.

If you want a one-page overview of what this is about, https://maex.me/2019/12/the-power-of-the-xdg-base-directory-specification/ seems simple.

ChrisW67

@Bengt-0 said in QSettings file location in Ubuntu ambiguous?:

~$ ls -ld /etc/xdg
drwxr-xr-x 12 root root 4096 Oct 6 21:07 /etc/xdg
~$ ls -ld /etc/xdg/xdg-Lubuntu
drwxr-xr-x 9 root root 4096 Oct 6 21:16 /etc/xdg/xdg-Lubuntu

OK, so neither location is writable to a non-root user. I assume your application is not running as root. When you ask Qt for:

QString path=QStandardPaths::writableLocation(QStandardPaths::AppConfigLocation);

it will not give you either of those locations because they are not writable.

I can see that my settings INI file is stored in /etc/xdg/myOrg/ as expected.

Your user-specific. modifiable application configuration files should be in:
QStandardPaths::ConfigLocation or QStandardPaths::AppConfigLocation (On Linux under ~/.config)

Along with my settings INI file, I want to store a log file at the same location.

That is very poor place to place a log file. For a system-wide tool the appropriate place would be /var/log (or use syslog). See Filesystem Hierarchy Standard (FHS) for more.

If your application log file is for user consumption, then it should go somewhere the user defines.
If your application log file is for debugging or similar use then you could use
QStandardPaths::AppDataLocation or QStandardPaths::AppLocalDataLocation (Windows)
If your application log file is for temporary purposes only then
QStandardPaths::TempLocation

Bengt 0

@ChrisW67
Chris,

I have a very good reason to put my log and config, and settings in a system-scope folder.
The software is driving a measurement system, to be configured and calibrated by a maintenance engineer as root user. The system config, calibration data and log of the accuracy are valuable and should not be risked of loss.

Actually, what I call a "log file" might not qualify exactly as a log file in the regular sense. It is more of a receipt of a calibration operation, describing parameters used and the result of the calibration. This is done on a monthly basis, and it is desirable to have long term a history available of the equipment performance. To my (limited) experience, /var/log can be a pretty wild and crowded place, important things might be lost there.

The system is used daily by regular non-root users, and the user-scope prefs are placed in a standard user pref location. If the software binaries are updated, the config, calibration data and calibration logs are not touched. If a regular user is making a serious mistake, the config and calibration data are still safe.
So my requirements are a bit special.

BN

Bengt 0

@JonB
Ok, so it is similar to the PATH variable.
Searching for available binary applications by walking they the PATH list is quite logical, but doing the same for config files is less obvious. At least to me, but I get the idea. Thanks.

ChrisW67

Alright. You want to use the fallback mechanism to achieve this.

When the application is run effectively as root and you go to save the global configuration, calibration data etc. then you should use QSettings(QSettings::SystemScope). In that case the settings will go under /etc/xdg by default. Write the shared settings into a group named e.g., "Global".

When the application is not run as root (the vast majority of time), open QSettings (QSettings::UserScope).
User-specific settings should be written to and read from a group called e.g., "User", to avoid any potential for collision.
When the user reads from group "Global" they will see the the central configuration file because of the fallback mechanism. The user mode code should never call setValue() on values in the "Global" group because that will create settings locally that will then mask the central ones.

#include <QCoreApplication>
#include <QSettings>
#include <QDateTime>
#include <QDebug>

#include <unistd.h>
#include <sys/types.h>


int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    QCoreApplication::setOrganizationName("Example");
    QCoreApplication::setOrganizationDomain("example.com");
    QCoreApplication::setApplicationName("Test");

    uid_t euid = geteuid();
    if (euid == 0) {
        QSettings settings(QSettings::SystemScope);
        qDebug() << "Running as root:" << settings.fileName();

        // System-wide settings should only ever be written in here
        settings.beginGroup("Global");
        settings.setValue("LastChange", QDateTime::currentDateTimeUtc().toString(Qt::ISODate));
        settings.setValue("Calibration", 36);
        settings.endGroup();
    }
    else {
          QSettings settings(QSettings::UserScope);
          qDebug() << "Running as user:" << settings.fileName();

          // System-wide settings are visible and read-only in here
          qDebug() << "Global last change:" << settings.value("Global/LastChange");
          qDebug() << "Global Calibration:" << settings.value("Global/Calibration");

          // User-specific settings can be read/write in here
          qDebug() << "User last change:" << settings.value("User/LastChange");
          qDebug() << "User fun count  :" << settings.value("User/AmountOfMeasurementFun");

          settings.beginGroup("User");
          settings.setValue("LastChange", QDateTime::currentDateTimeUtc().toString(Qt::ISODate));
          int funCount = settings.value("AmountOfMeasurementFun", 0).toInt();
          settings.setValue("AmountOfMeasurementFun", ++funCount);
          settings.endGroup();
    }

    return 0;
}

Tests:

$ ls -ld /etc/xdg/Example ~/.config/Example
ls: cannot access '/etc/xdg/Example': No such file or directory
ls: cannot access '/home/chrisw/.config/Example': No such file or directory
$ ./test 
Running as user: "/home/chrisw/.config/Example/Test.conf"
Global last change: QVariant(Invalid)
Global Calibration: QVariant(Invalid)
User last change: QVariant(Invalid)
User fun count  : QVariant(Invalid)

There was no system config but a user config has now been written:

$ cat /home/chrisw/.config/Example/Test.conf
[User]
AmountOfMeasurementFun=1
LastChange=2022-10-08T02:22:24Z

Now a calibration session is run as root:

$ sudo ./test
Running as root: "/etc/xdg/Example/Test.conf"
$ cat /home/chrisw/.config/Example/Test.conf
[User]
AmountOfMeasurementFun=1
LastChange=2022-10-08T02:22:24Z
$ cat /etc/xdg/Example/Test.conf 
[Global]
Calibration=36
LastChange=2022-10-08T02:22:48Z

The user settings unchanged but now the system settings exist, the user sees them.

$ ./test 
Running as user: "/home/chrisw/.config/Example/Test.conf"
Global last change: QVariant(QString, "2022-10-08T02:22:48Z")
Global Calibration: QVariant(QString, "36")
User last change: QVariant(QString, "2022-10-08T02:22:24Z")
User fun count  : QVariant(QString, "1")
$ ./test 
Running as user: "/home/chrisw/.config/Example/Test.conf"
Global last change: QVariant(QString, "2022-10-08T02:22:48Z")
Global Calibration: QVariant(QString, "36")
User last change: QVariant(QString, "2022-10-08T02:23:08Z")
User fun count  : QVariant(QString, "2")

The global settings are never in the user's configuration file.

$ cat /home/chrisw/.config/Example/Test.conf
[User]
AmountOfMeasurementFun=3
LastChange=2022-10-08T02:23:29Z

JonB

@Bengt-0
Firstly, please read through @ChrisW67's very thorough sample code in his last post.

Going back to your original question/"bad" behaviour:

I can see that my settings INI file is stored in /etc/xdg/myOrg/ as expected.

Along with my settings INI file, I want to store a log file at the same location.

In a different section of my program, I use QSettings::fileName() to find the path to the log file.

However, the file path from ::fileName is instead /etc/xdg/xdg-Lubuntu/myOrg, not /etc/xdg/myOrg.

I do not get this. In particular I do not get your "I can see that my settings INI file is stored in /etc/xdg/myOrg/ as expected.". And I don't know how exactly this comes about for you, since you showed

~$ ls -ld /etc/xdg
drwxr-xr-x 12 root root 4096 Oct 6 21:07 /etc/xdg

Since /etc/xdg is writable only by root, how did you get /etc/xdg/myOrg/ and what are its permissions/who is it writable by?

Here is the test code I used:

#include <QCoreApplication>
#include <QDebug>
#include <QSettings>

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    QCoreApplication::setOrganizationName("Example");
    QCoreApplication::setOrganizationDomain("example.com");
    QCoreApplication::setApplicationName("Settings");

    QSettings userSettings(QSettings::UserScope, "Example", "Settings");
    qDebug() << userSettings.fileName();
    userSettings.setValue("abc", "def");
    userSettings.sync();
    qDebug() << userSettings.status();

    QSettings systemSettings(QSettings::SystemScope, "Example", "Settings");
    qDebug() << systemSettings.fileName();
    systemSettings.setValue("abc", "def");
    systemSettings.sync();
    qDebug() << systemSettings.status();

    return a.exec();
}

[I have to use a slightly different QSettings constructor from you since I am on Qt 5.12, but it's the same effect.] Running as me, not root (is that your case too?). And my output is:

"/home/jon/.config/Example/Settings.conf"
QSettings::NoError
"/etc/xdg/xdg-ubuntu/Example/Settings.conf"
QSettings::AccessError

just as I would expect.

Bengt 0

@JonB
Jon,

I sense some possible miscommunication here.

First, the listing of directory content.
I am using "ls-la" and not "ls -ld":

$ ls -la /etc/xdg/
total 76
drwxr-xr-x  13 root root  4096 Oct  8 12:15 .
drwxr-xr-x 130 root root 12288 Oct  7 08:32 ..
-rw-r--r--   1 root root    86 Mar  5  2022 accept-languages.codes
drwxr-xr-x   2 root root  4096 Oct  7 08:00 autostart
drwxr-xr-x   2 root root  4096 Oct  8 11:53 Example
-rw-r--r--   1 root root   153 Mar  5  2022 kshorturifilterrc
drwxr-xr-x   2 root root  4096 Aug  9 13:50 libfm
drwxr-xr-x   3 root root  4096 Aug  9 13:49 lxqt
drwxr-xr-x   2 root root  4096 Aug  9 13:51 menus
drwxr-xr-x   2 root root  4096 Oct  8 12:37 MYORG
drwxr-xr-x   2 root root  4096 Aug  9 13:50 openbox
drwxr-xr-x   2 root root  4096 Aug  9 13:47 systemd
drwxr-xr-x   2 root root  4096 Aug  9 13:50 ui
-rw-r--r--   1 root root   414 Mar 25  2022 user-dirs.conf
-rw-r--r--   1 root root   418 Mar 25  2022 user-dirs.defaults
drwxr-xr-x   9 root root  4096 Aug  9 13:51 xdg-Lubuntu
drwxr-xr-x   2 root root  4096 Aug  9 13:51 Xwayland-session.d

$ ls -la /etc/xdg/MYORG
total 12
drwxr-xr-x  2 root root 4096 Oct  8 12:37 .
drwxr-xr-x 13 root root 4096 Oct  8 12:15 ..
-rw-r--r--  1 root root  227 Oct  8 12:37 MYAPP.ini

$ ls -la /etc/xdg/xdg-Lubuntu/
total 48
drwxr-xr-x  9 root root 4096 Aug  9 13:51 .
drwxr-xr-x 13 root root 4096 Oct  8 12:15 ..
drwxr-xr-x  2 root root 4096 Aug  9 13:51 autostart
drwxr-xr-x  2 root root 4096 Aug  9 13:51 libfm
drwxr-xr-x  2 root root 4096 Aug  9 13:51 lxqt
-rw-r--r--  1 root root 8468 Nov 14  2021 mimeapps.list
drwxr-xr-x  2 root root 4096 Aug  9 13:51 nm-tray
drwxr-xr-x  2 root root 4096 Aug  9 13:51 openbox
drwxr-xr-x  3 root root 4096 Aug  9 13:49 pcmanfm-qt
drwxr-xr-x  2 root root 4096 Aug  9 13:51 qterminal.org

Here you can see that MYORG resides in /etc/xdg, and MYAPP.ini is in /etc/xdg/MYORG. /etc/xdg/xdg-Lubuntu does not contain any of my files.

Now to the sample code, and this is interesting. Thanks for supplying the sample code, both of you, you were quicker than me.

The sample code from @ChrisW67 works correctly. I tried it and got the same result.
However, your code (JonB) reproduces my problem if I try it, and you also write this in your post.

"/home/jon/.config/Example/Settings.conf"
QSettings::NoError
"/etc/xdg/xdg-ubuntu/Example/Settings.conf"
QSettings::AccessError

You seem to think this is ok and expected, you need to explain why to me.

I would expect qDebug() << systemSettings.fileName(); to give "/etc/xdg/Example/Settings.conf".

There is something I do not understand here.

BN

ChrisW67

@Bengt-0 said in QSettings file location in Ubuntu ambiguous?:

You seem to think this is ok and expected, you need to explain why to me.

The test program is running as an unprivileged user. As a result it cannot write into /etc/xdg to create Example/Settings.conf and you get the Access error.

On my Kubuntu machine an unprivileged user opening QSettings::SystemScope actually does get a writable path, but it is under ~/.config/kdedefaults not /etc/xdg (where it would fail). This is likely an environmental difference (i.e. XDG_CONFIG_DIRS has been modified by KDE to include the writable path first).

Bengt 0

@JonB

Got it now.

If I try to access read/write SystemScope prefs as root, the app looks in /etc/xdg/MYORG.
If I try to access readonly SystemScope prefs as user, the app looks in /etc/xdg/xdg-Lubuntu/MYORG.

Is this normal and expected?

JonB

@Bengt-0
You show

drwxr-xr-x 2 root root 4096 Oct 8 11:53 Example
-rw-r--r-- 1 root root 227 Oct 8 12:37 MYAPP.ini

Both of these are owned by root, and in directories only writable by root. I stated:

Running as me, not root (is that your case too?)

Are you running as root?? Else I'm not sure how you have permission to write to either of these?

I would expect qDebug() << systemSettings.fileName(); to give "/etc/xdg/Example/Settings.conf"

I would not, because /etc/xdg is not writable by me (user jon), so Qt (or XDG) is not going to pick that as a suitable location to save/write a config file.

You need to be 100% explicit about:

Who you are running your application as?
Are you indeed interested in QSettings::SystemScope, not UserScope?

[This was in answer to your earlier post, you may be beginning to clarify in your latest post.]

Bengt 0

@JonB

We have a difficulty with timing right now..:)

What I am interested in:

I want to WRITE the settings as root, System Scope.
I want to READONLY the same SystemScope settings as user.

JonB

@Bengt-0 said in QSettings file location in Ubuntu ambiguous?:

If I try to access read/write SystemScope prefs as root, the app looks in /etc/xdg/MYORG.

Yes, and no. Looking in etc/xdg is good for root. But since /etc/xdg/xdg-Lubuntu is first in your XDG_CONFIG_DIRS I would actually expect it to pick /etc/xdg/xdg-Lubuntu/myOrg over /etc/xdg/myOrg. Assuming your /etc/xdg/xdg-Lubuntu directory exists, remember my equivalent /etc/xdg/xdg-ubuntu does not exist. It might depend on whether your /etc/xdg/xdg-Lubuntu/myOrg or /etc/xdg/myOrg directories already exist, I don't know.

If I try to access readonly SystemScope prefs as user, the app looks in /etc/xdg/xdg-Lubuntu/MYORG.

I'm sorry, but now we are all over the place. I'm lost as to what does what when. It may depend on whether MYORG directory yet exists, and/or whether the ini/conf file is already in there, and writeability. We do not have the same starting set up. If I were you I would start by deleting the file at /etc/xdg/myOrg/MYAPP.ini so that you are "clean".

P.S.

but it is under ~/.config/kdedefaults not /etc/xdg (where it would fail). This is likely an environmental difference (i.e. XDG_CONFIG_DIRS has been modified by KDE to include the writable path first).

Nope, I don't think KDE is going to modify anything XDG. I would think either QSettings or whatever code of theirs is picking a writable location.

I want to WRITE the settings as root, System Scope.

I want to READONLY the same SystemScope settings as user

QSettings normally expects a writable file/location. I'm not sure how/whether to tell it you don't want that for plain user. Don't forget if required you can pass a full path for the file to QSettings, so you can control it. E.g. maybe you will have to try to find the file saved by root to pass for the user? The whole idea of relying on user being able to get returned the same file location as where root might have saved by relying on QSettings behaving "similar" for both sounds a warning bell for me.

Bengt 0

Can we agree on what I am trying to achieve?

Writing SystemScope prefs as root, reading the same SystemScope prefs user?

JonB

@Bengt-0
Read the final paragraph of what I have just written. I'm not going to say I know how QSettings works enough to give you the guarantee you are looking for when run as different users and letting QSettings pick paths. At this point if you want to be sure/have a definitive answer you should look at the Qt source code, else we are both whistling in the wind.

Bengt 0

Ok, it seems I have to review my intentions.

I should have started by informing that this application was originally written for Windows, and I am now trying to make it work for linux. I was assuming the cross-platform properties of Qt5 to work more or less without (structural) modifications, which now appear not to be entirely true.
Under Windows, UserScope prefs are placed at ~\AppData\Roaming\MYORG, and SystemScope prefs are placed at C:\ProgramData\MYORG. Read/Write access is determined by being Administrator or not. Works fine.

Under linux, the SystemScope prefs are placed at /etc/xdg by default for root and /etc/xdg/xdg-Lubuntu by default for user, not an ideal solution.

If I am going to change this by an explicit SystemScope path to something of my own, what would you suggest? Keep /etc/xdg, or something else entirely that would be more according to linux standards?

I guess that it is very obvious to everyone reading these posts that I am although an engineer, I am not a SOFTWARE engineer. Advice would be very helpful.

JonB

@Bengt-0
I think this is a better question :)

I want to be 100% clear:

I want to WRITE the settings as root, System Scope.

I want to READONLY the same SystemScope settings as user

This is just one shared file, written by root, read but not never written by user. When you started talking about "UserScope prefs are placed at ~\AppData\Roaming\MYORG", that is quite separate and not what you are asking about here, right? That will just be whatever UserScope under Linux too.

So for the SystemScope you want best advice where under Linux to place a configuration file for shared user readership. What sort of data are you going to share/put in the system QSettings file? It it stuff to do with, say, the UI, or nothing to do with UIs or desktops?

Bengt 0

@JonB

Your are correct, the UserScope prefs location is not a serious issue.
But in principle yes, the UserScope prefs are different for root and user, so the window location and size will be different, and some of the other user settings too. But this can be handled and is not a showstopper.

Here is an example of the SystemScope INI file:

[General]
SerialPort=COM3
NegateX=0
NegateY=0
SwapXY=0

[Settings]
SYSTYPE-1\TransX=766.662
SYSTYPE-1\TransY=381.933
SYSTYPE-1\ScaleX=1.000049352646
SYSTYPE-1\ScaleY=0.999990820885
SYSTYPE-1\SkewX=0.000283443864
SYSTYPE-1\SkewY=-0.000374994561
SYSTYPE-1\CalDate=

These parameters are only for internal use, nothing to do with the GUI or desktop.

Bengt 0

Thanks for a valuable discussion.
I managed to solve it by using a wrapper startup script with:

unset   XDG_CONFIG_DIRS

before the actual start of the executable. Then the path defaults to /etc/xdg both for user and root.
This is described in the Qt docs, and also in this thread.
But, unfortunately, reading and understanding are not the same thing.
At least not for me.

BN