Unicode on ES, files aren't shown in the correct encoding

QtEmbeddor

Hi,

I'm running a QML-Application on an embedded Linux.
It provides file processing using QFile, QDir and QFileSystemWatcher (creating, reading, browsing). On Ubuntu Unicode works fine for the files. Creating different files work fine including foreign letters like Russian and umlauts (жю,äöü) but on the ES. it doesn't.

Files are not displayed correct, although Unicode in general works in the QT-App (the UI is available in Russian language)
Files created get file names like "201Ð¼Ð°Ð½Ñ.txt"
Existing files are displayed like "201?????.txt"
Via SSH files are displayed by ls like "201?????.txt"
Via SSH the completion by <Tabkey> shows files correctly like "201датер.txt"
Via SSH typing echo <a string> shows the string my remote terminal like "???" the output of echo fine like датер. vi editor shows ..... and cat outputs датер.

perhaps its not a QT-problem and more a Linux (file system) problem.
No

 LANG=en_US.UTF-8

or similar is set but if I set it and restart the app nothing changes.
Is a package missing that QT-Applications needs to work proper?

Does someone have a solution or suggestion for that kind of problem?
If that problem is unrelated to QT forgive me.

SGaist

Hi and welcome to devnet,

What is "ES" ? What Linux distribution are you running on it ? Do you have the same fonts as your desktop machine available on it ?

QtEmbeddor

ES=Embedded System. A Linux from scratch is running there so no known distro.
Fonts are the same as on the desktop machine (Fira Sans). Fonts shouldn't be the problem since the whole gui is working in Russian and other languages having umlauts.

SGaist

Sorry, I misunderstood part of the problem.

Do you know what encoding was used for the files you are trying to read ? Are they in UTF-8 or something else ?

QtEmbeddor

It concerns only the encoding of file names.

files created using the QT-Application contain "?" instead of the not ASCII-Characters (really question marks). They are saved as names containing question marks.
files created in the terminal created like echo кзж > кзж are shown like "Ð¼Ð" in the QT-Application, as "???" by ls but auto completed in Cyrillic in console

The Encoding inside the files works fine!

Probably there is a library missing but which? I found this: The Program Uses or Creates Filenames in the Wrong Encoding which doesn't helped me but maybe concerns the same problem.

SGaist

Can you share a small code sample creating a file with a failing name ? It will be easier to test.

J.Hilk

@QtEmbeddor
I'm assuming your local Codec is set wrong.

E.g QTextStream uses by default the QTextCodec::codecForLocale(), but automatic unicode detection should be enabled as well.

You could force unicode, to make sure:

//Taken directely from the docu:
QByteArray encodedString = "...";
QTextCodec *codec = QTextCodec::codecForName("UTF-16");
QString string = codec->toUnicode(encodedString);

QtEmbeddor

The solution to that problem is to insert the following call to the constructor of the file processing class:

QTextCodec::setCodecForLocale(QTextCodec::codecForName("UTF-8"));

Concerning the shell:
I use BusyBox (its light) which doesn't provide locale and the environment variables like LANG=en_US.UTF-8.
For compiling BusyBox configurations can or have to be given containing

CONFIG_LAST_SUPPORTED_WCHAR=0

the number is the last displayable number of the Unicode list, 0 signifies that all characters shall be displayed, it was set to 767 which includes west European characters. The excluded characters are replaced by the character defined in the same configs, in my case it is "?".