Qt 6.10 and text encodings

[Christian Ehrlicher]

I've updated https://wiki.qt.io/Compiling-ICU-with-MinGW - Qt6 can be compiled with icu provided by mingw.

[AndyBrice]

@cristian-adam Thansk for that.

There is quite a big difference in the encoding supported by Qt 5 and Qt 6. As far as I can make out:

Supported by Qt 5 and Qt 6.10:

"Big5"
"Big5-HKSCS"
"EUC-JP"
"EUC-KR"
"GB18030"
"GB2312"
"GBK"
"IBM850"
"IBM866"
"ISO-2022-JP"
"ISO-8859-1"
"ISO-8859-10"
"ISO-8859-13"
"ISO-8859-14"
"ISO-8859-15"
"ISO-8859-2"
"ISO-8859-3"
"ISO-8859-4"
"ISO-8859-5"
"ISO-8859-6"
"ISO-8859-7"
"ISO-8859-8"
"ISO-8859-9"
"KOI8-R"
"KOI8-U"
"Shift_JIS"
"TIS-620"
"UTF-16"
"UTF-16BE"
"UTF-16LE"
"UTF-32"
"UTF-32BE"
"UTF-32LE"
"UTF-8"
"hp-roman8"
"macintosh"
"windows-1250"
"windows-1251"
"windows-1252"
"windows-1253"
"windows-1254"
"windows-1255"
"windows-1256"
"windows-1257"
"windows-1258"

Supported by Qt 5 only:

"ASMO-708"
"Apple Roman"
"Big5-ETen"
"CP1250"
"CP1251"
"CP1252"
"CP1253"
"CP1254"
"CP1255"
"CP1256"
"CP1257"
"CP1258"
"CP819"
"CP850"
"CP866"
"CP874"
"CP936"
"CP949"
"CP950"
"ECMA-114"
"ECMA-118"
"IBM819"
"IBM874"
"ISO 8859-11"
"ISO 8859-8-I"
"ISO-8859-10:1992"
"ISO-8859-16"
"ISO-8859-6-I"
"JIS7"
"KOI8-RU"
"MS936"
"MS_Kanji"
"MacRoman"
"SJIS"
"System"
"TSCII"
"WINSAMI2"
"WS2"
"arabic"
"csHPRoman8"
"csIBM866"
"csISOLatin1"
"csISOLatin2"
"csISOLatin3"
"csISOLatin4"
"csISOLatin5"
"csISOLatin6"
"csISOLatinArabic"
"csISOLatinCyrillic"
"csISOLatinGreek"
"csISOLatinHebrew"
"csKOI8R"
"csPC850Multilingual"
"cyrillic"
"greek"
"hebrew"
"iscii-bng"
"iscii-dev"
"iscii-gjr"
"iscii-knd"
"iscii-mlm"
"iscii-ori"
"iscii-pnj"
"iscii-tlg"
"iscii-tml"
"iso-celtic"
"iso-ir-100"
"iso-ir-101"
"iso-ir-109"
"iso-ir-110"
"iso-ir-126"
"iso-ir-127"
"iso-ir-138"
"iso-ir-144"
"iso-ir-148"
"iso-ir-157"
"iso-ir-199"
"iso-ir-226"
"latin1"
"latin10"
"latin2"
"latin3"
"latin4"
"latin5"
"latin6"
"latin8"
"latin9"
"roman8"
"windows-936"
"windows-949"

Supported by Qt 6 only:

"Adobe-Standard-Encoding"
"BOCU-1"
"CESU-8"
"GB_2312-80"
"HZ-GB-2312"
"IBM-Thai"
"IBM00858"
"IBM01140"
"IBM01141"
"IBM01142"
"IBM01143"
"IBM01144"
"IBM01145"
"IBM01146"
"IBM01147"
"IBM01148"
"IBM01149"
"IBM037"
"IBM1026"
"IBM1047"
"IBM273"
"IBM277"
"IBM278"
"IBM280"
"IBM284"
"IBM285"
"IBM290"
"IBM297"
"IBM420"
"IBM424"
"IBM437"
"IBM500"
"IBM775"
"IBM852"
"IBM855"
"IBM857"
"IBM860"
"IBM861"
"IBM862"
"IBM863"
"IBM864"
"IBM865"
"IBM868"
"IBM869"
"IBM870"
"IBM871"
"IBM918"
"IMAP-mailbox-name"
"ISCII,version=0"
"ISCII,version=1"
"ISCII,version=2"
"ISCII,version=3"
"ISCII,version=4"
"ISCII,version=5"
"ISCII,version=6"
"ISCII,version=7"
"ISCII,version=8"
"ISO-2022-CN"
"ISO-2022-CN-EXT"
"ISO-2022-JP-1"
"ISO-2022-JP-2"
"ISO-2022-KR"
"ISO_2022,locale=ja,version=3"
"ISO_2022,locale=ja,version=4"
"ISO_2022,locale=ko,version=1"
"ISO_2022,locale=zh,version=2"
"KSC_5601"
"LMBCS-1"
"Locale"
"SCSU"
"US-ASCII"
"UTF-16,version=1"
"UTF-16,version=2"
"UTF-16BE,version=1"
"UTF-16LE,version=1"
"UTF-7"
"UTF16_OppositeEndian"
"UTF16_PlatformEndian"
"UTF32_OppositeEndian"
"UTF32_PlatformEndian"
"cp1363"
"cp851"
"ebcdic-xml-us"
"euc-tw-2014"
"gsm-03.38-2009"
"ibm-1006_P100-1995"
"ibm-1025_P100-1995"
"ibm-1047_P100-1995,swaplfnl"
"ibm-1097_P100-1995"
"ibm-1098_P100-1995"
"ibm-1112_P100-1995"
"ibm-1122_P100-1999"
"ibm-1123_P100-1995"
"ibm-1124_P100-1996"
"ibm-1125_P100-1997"
"ibm-1129_P100-1997"
"ibm-1130_P100-1997"
"ibm-1131_P100-1997"
"ibm-1132_P100-1998"
"ibm-1133_P100-1997"
"ibm-1137_P100-1999"
"ibm-1140_P100-1997,swaplfnl"
"ibm-1141_P100-1997,swaplfnl"
"ibm-1142_P100-1997,swaplfnl"
"ibm-1143_P100-1997,swaplfnl"
"ibm-1144_P100-1997,swaplfnl"
"ibm-1145_P100-1997,swaplfnl"
"ibm-1146_P100-1997,swaplfnl"
"ibm-1147_P100-1997,swaplfnl"
"ibm-1148_P100-1997,swaplfnl"
"ibm-1149_P100-1997,swaplfnl"
"ibm-1153_P100-1999"
"ibm-1153_P100-1999,swaplfnl"
"ibm-1154_P100-1999"
"ibm-1155_P100-1999"
"ibm-1156_P100-1999"
"ibm-1157_P100-1999"
"ibm-1158_P100-1999"
"ibm-1160_P100-1999"
"ibm-1162_P100-1999"
"ibm-1164_P100-1999"
"ibm-1250_P100-1995"
"ibm-1251_P100-1995"
"ibm-1252_P100-2000"
"ibm-1253_P100-1995"
"ibm-1254_P100-1995"
"ibm-1255_P100-1995"
"ibm-1256_P110-1997"
"ibm-1257_P100-1995"
"ibm-1258_P100-1997"
"ibm-12712_P100-1998"
"ibm-12712_P100-1998,swaplfnl"
"ibm-1363_P110-1997"
"ibm-1364_P110-2007"
"ibm-1371_P100-1999"
"ibm-1373_P100-2002"
"ibm-1386_P100-2001"
"ibm-1388_P103-2001"
"ibm-1390_P110-2003"
"ibm-1399_P110-2003"
"ibm-16684_P110-2003"
"ibm-16804_X110-1999"
"ibm-16804_X110-1999,swaplfnl"
"ibm-33722_P120-1999"
"ibm-33722_P12A_P12A-2009_U2"
"ibm-37_P100-1995,swaplfnl"
"ibm-4517_P100-2005"
"ibm-4899_P100-1998"
"ibm-4909_P100-1999"
"ibm-4971_P100-1999"
"ibm-5123_P100-1999"
"ibm-5351_P100-1998"
"ibm-5352_P100-1998"
"ibm-5353_P100-1998"
"ibm-5471_P100-2006"
"ibm-720_P100-1997"
"ibm-737_P100-1997"
"ibm-803_P100-1999"
"ibm-813_P100-1995"
"ibm-8482_P100-1999"
"ibm-856_P100-1995"
"ibm-867_P100-1998"
"ibm-875_P100-1995"
"ibm-901_P100-1999"
"ibm-902_P100-1999"
"ibm-9067_X100-2005"
"ibm-916_P100-1995"
"ibm-922_P100-1999"
"ibm-930_P120-1999"
"ibm-933_P110-1995"
"ibm-935_P110-1999"
"ibm-937_P110-1999"
"ibm-939_P120-1999"
"ibm-942_P12A-1999"
"ibm-943_P130-1999"
"ibm-949_P110-1999"
"ibm-949_P11A-1999"
"ibm-950_P110-1999"
"ibm-954_P101-2007"
"ibm-964_P110-1999"
"ibm-971_P100-1995"
"iso-8859_11-2001"
"windows-874-2000"
"x-mac-centraleurroman"
"x-mac-cyrillic"
"x-mac-greek"
"x-mac-turkish"
"x11-compound-text"

Have some of the encodings been renamed between Qt 5 and Qt 6?

[AndyBrice]

BTW I didn't find any mention of the addition of more encodings in the Qt 6.10 release notes.

Anyone know if they have also been added back into Mac in Qt 6.10? My dev Mac is on macOS 12 and Qt 6.10 doesn't seem to support macOS 12 (it doesn't show up in the kit list, even when downloaded by the maintenance tool).

[cristian-adam]

@AndyBrice said in Qt 6.10 and text encodings:

Have some of the encodings been renamed between Qt 5 and Qt 6?

IIRC Qt5 has its own Codecs implementation, now Qt6 uses ICU. They are two different implementations.

[cristian-adam]

@AndyBrice said in Qt 6.10 and text encodings:

BTW I didn't find any mention of the addition of more encodings in the Qt 6.10 release notes.

Anyone know if they have also been added back into Mac in Qt 6.10? My dev Mac is on macOS 12 and Qt 6.10 doesn't seem to support macOS 12 (it doesn't show up in the kit list, even when downloaded by the maintenance tool).

Just tested on macOS with Qt 6.10:

Starting /Users/cristian/Projects/C++/TextEncodings/build/Qt_6_10_0_for_macOS-Debug/TextEncodings...
"UTF-8"
"UTF-16"
"UTF-16LE"
"UTF-16BE"
"UTF-32"
"UTF-32LE"
"UTF-32BE"
"ISO-8859-1"

Oh noes! macOS is also not supporting ICU! 🤦🏻‍♂️

[AndyBrice]

@cristian-adam Thanks for the explanation. I don't suppose anyone has written down how one maps onto the other?

[AndyBrice]

@cristian-adam said in Qt 6.10 and text encodings:

Oh noes! macOS is also not supporting ICU!

That is less than useful when you are trying to write a Windows/Mac cross-platform application!

I want to move Easy Data Transform from Qt 5 to Qt 6. But currently I can't without going from supporting ~60 text encodings to 8.

[cristian-adam]

@AndyBrice said in Qt 6.10 and text encodings:

@cristian-adam said in Qt 6.10 and text encodings:

Oh noes! macOS is also not supporting ICU!

That is less than useful when you are trying to write a Windows/Mac cross-platform application!

I want to move Easy Data Transform from Qt 5 to Qt 6. But currently I can't without going from supporting ~60 text encodings to 8.

The feature is there, it's just the official build doesn't provision ICU on all platforms.

If you build Qt yourself using a package manager like vcpkg or conan you can get a consistent feature set.

[AndyBrice]

@cristian-adam I used to build Qt myself, many years ago. But it just got too difficult.

Is ICU going to supported in a future Mac binary release?

[cristian-adam]

@AndyBrice said in Qt 6.10 and text encodings:

@cristian-adam I used to build Qt myself, many years ago. But it just got too difficult.

Is ICU going to supported in a future Mac binary release?

No idea. I've opened up https://bugreports.qt.io/browse/QTBUG-141087