Can I read/write Windows-1252 and other legacy encodings in Qt 6?

[Bonnie]

From QStringConverter::encodingForName, it says

Such a name may, none the less, be accepted by the QStringConverter constructor when Qt is built with ICU, if ICU provides a converter with the given name.

So it should be possible to load an icu-supported codec name. (Seems to start from 6.6 according to this SO post.)
But I'm not sure if the prebuilt Qt is built with ICU or not.

[ankou29666]

@AndyBrice said in Can I read/write Windows-1252 and other legacy encodings in Qt 6?:

My customers want to be able to read and write using encodings such as Windows-1252 and Windows-1256. This is stopping me moving to Qt 6.

If I can understand the need of reading existing data in old formats, however, is there really a point writing to those old formats ?

[aha_1980]

@ankou29666

If I can understand the need of reading existing data in old formats, however, is there really a point writing to those old formats ?

Interoperability with existing, old applications?

[AndyBrice]

@ankou29666 said in Can I read/write Windows-1252 and other legacy encodings in Qt 6?:

If I can understand the need of reading existing data in old formats, however, is there really a point writing to those old formats ?

In an ideal world, no. But people have to work with all sorts of legacy systems.

[AndyBrice]

@Bonnie said in Can I read/write Windows-1252 and other legacy encodings in Qt 6?:

So it should be possible to load an icu-supported codec name. (Seems to start from 6.6 according to this SO post.)
But I'm not sure if the prebuilt Qt is built with ICU or not.

As far as I can make out, it isn't possible to access these additional Codecs from the Qt 6.7 or 6.8 binaries.

Years ago, I used to build my own Qt binaries from source. But it just got too difficult.

[AndyBrice]

I guess my other option is to build a command line encoding converter in Qt 5 and call it from my Qt 6 application. Hardly ideal though.

[ankou29666]

seems like the QTextCodec in Qt5 compatibility module supports Win1250 to 1258 endodings.
https://doc.qt.io/qt-6/qtextcodec.html

[Bonnie]

@AndyBrice Or you can link to icu and use its api your self just like what Qt did in its internal codes, or even use other thirdparty codec libraries.

[AndyBrice]

@ankou29666 said in Can I read/write Windows-1252 and other legacy encodings in Qt 6?:

seems like the QTextCodec in Qt5 compatibility module supports Win1250 to 1258 endodings.
https://doc.qt.io/qt-6/qtextcodec.html

Ok, I didn't spot that. Thanks.

So I can install handle these encodings, but to read windows-1252 encoding where I used to do this in Qt 5:

QFile f( path );
if ( f.open( QIODevice::ReadOnly ) )
{
  QTextStream t( &f );
  QTextCodec* codec = QTextCodec::codecForName( "windows-1252" );
  t.setCodec( codec );
  ...
}

I have to do this in Qt 6:

QByteArray encodedString = "..."; // read from file
QTextCodec* codec = QTextCodec::codecForName("windows-1252");
QString unencodedString = codec->toUnicode(encodedString);

Is that right?

[IgKh]

If willing to use a sub process, you can always use a iconv binary to pre/post process input or output to/from UTF-16. There are many ways to tackle this issue, which is probably part of the reason no one was motivated enough so far to push the patch to extend Qt 6's QTextCodec to the finish line.

[AndyBrice]

@Bonnie said in Can I read/write Windows-1252 and other legacy encodings in Qt 6?:

Or you can link to icu and use its api your self just like what Qt did in its internal codes, or even use other thirdparty codec libraries.

Are their prebuilt ICU binaries for Windows and Mac? I had a quick look on https://unicode-org.github.io/icu/, but didn't see them.

Do you know what the licensing of the binaries is? If they are GPL, I won't be able to use them in my commercial product.

[AndyBrice]

@IgKh said in Can I read/write Windows-1252 and other legacy encodings in Qt 6?:

If willing to use a sub process, you can always use a iconv binary to pre/post process input or output to/from UTF-16.

Is iconv related to the ICU libraries, or completely different?

[IgKh]

@AndyBrice iconv isn't related to ICU, it is a very old POSIX API and a corresponding CLI binary that's included in every UNIX-like/Linux system and it is not hard to find compatible Windows versions of it. Usually can work with any text encoding ever known to mankind.

It can be integrated using QProcess, i.e. something like:

QProcess* proc = new QProcess(parent);
proc->setStandardInputFile("path/to/input/file");
proc->start("path/to/iconv", QStringList() << "-f" << "WINDOWS-1252" << "-t" << "UTF16");

A then the QProcess can be used as source device for QTextStream, since it is a kind of QIODevice. Likewise for the output.

[AndyBrice]

@IgKh Ok, thanks for the explanation.

[SimonSchroeder]

@AndyBrice said in Can I read/write Windows-1252 and other legacy encodings in Qt 6?:

Are their prebuilt ICU binaries for Windows and Mac? I had a quick look on https://unicode-org.github.io/icu/, but didn't see them.

Binaries are located on their GitHub page under release: https://github.com/unicode-org/icu/releases/tag/release-76-rc. Upon a quick glance I'm not sure if any of these are for macOS, though.

@AndyBrice said in Can I read/write Windows-1252 and other legacy encodings in Qt 6?:

Do you know what the licensing of the binaries is? If they are GPL, I won't be able to use them in my commercial product.

They have a lilst of all the licenses (including 3rd party) that apply: https://github.com/unicode-org/icu?tab=License-1-ov-file. ICU itself seems to be very permissive. Some of the 3rd party libs seem to require a mention with their copyright notice. Overall it should be useable for commercial products.

[AndyBrice]

Thanks, Simon.