QLabel setText using a utf-8 string is rendered with Latin1 encoding

Jessevg

This problem presents itself on our embedded device, if we pass utf-8 strings to the setText of a QLabel it will render for example ° as 2 Latin1 characters, however if I make a small gui on my desktop and do setText there with a utf-8 string it will show the ° sign correctly.

There must be some way for Qt to detect the kind of text encoding used or a way to set it, but I can't figure it out, anyone knows what's up here?

The embedded version uses Qt 6.4.0 and sets a fixed font to DejaVu Sans from here: https://github.com/dejavu-fonts/dejavu-fonts

kkoehne

Which Qt version are you targetting?

Jessevg

Desktop version that uses utf8 correctly is Qt 6.3.0/6.3.2, on Windows as well as my Ubuntu this works, our embedded version is 6.4.0 (plain embedded Linux built with buildroot) I can provide the GUI/Widgets related cmake config options for our embedded Qt build if this would help

Christian Ehrlicher

So in your non-working environment - does your compiler there use utf-8 as input encoding and is the locale also utf-8?

Rule of thumb: don't use non-ascii chars in code, use QTranslator.

Jessevg

@Christian-Ehrlicher

The locale I'm not sure, it's in buildroot, but I'll put it on the mailing list if someone there can help me along.
As for the actual source file this originates from, everything worked when the file was encoded as ISO 8859-1, but I removed some code there, which forced an UTF-8 change as our source control system expects every new/changed file to be UTF-8 as a new standard, and that's when things started to break, compilation is standard gcc cross compilation, and this one seems to do things correct, as it generates the correct char array I provide to my QString, I did check that, it's just passing that QString to the gui element that I need to be able to do with UTF-8

kkoehne

From Qt 6 onwards, Qt treats all source code as UTF-8. And QString internally has always used UTF-16. So if you have checked that the QString actually has the correct content, it's most likely a font rendering problem.

But have you really double-checked the contents of the QString, e.g. with

QChar *data = str.data();
while (!data->isNull()) {
    qDebug() << data->unicode();
    ++data;
}

?

Jessevg

@kkoehne

That was a good tip, I was just looking at raw data but my unicode char was split up into 2 QChars, explaining the double chars I saw.
This however led me to the real culprit of my troubles, QString::fromLocal8bit.

The incorrect string came from such a call with a char array, and the documentation led me to believe this was fully equivalent to a QString::fromUtf8 call as it states "On Unix systems this is equivalen to fromUtf8(), on Windows the systems current code page is being used."

However looking at the implementation this doesn't seem to be the case because QStringDecoder with System encoding is used which again states it'll be UTF-8 on a unix system, but this doesn't seem to be the case for me at first glance and I can't immediately find it in the implementation either.

I will investigate further and provide an appropriate bug report if needed depending on what I find.

Thanks for the help.

Christian Ehrlicher

@Jessevg said in QLabel setText using a utf-8 string is rendered with Latin1 encoding:

QStringDecoder with System encoding is used which again states it'll be UTF-8 on a unix system

QStringDecoder::System always assumes that it's utf-8, see qstringconvert_p.h / QLocal8Bit implementation for non-windows builds. Therefore when your locale on your target system is not utf-8 the stuff will get messed up.