QODBC and UTF-8 issues

mjsurette

I am trying to transfer data from Quickbooks to a Postgres database. Both are encoded in UTF-8, but Qt is interpreting the Quickbooks data as UTF-16, causing translation issues with non-English characters. If I use soci and std::string, I have no issue, but would much rather use the Qt database system.

Is there a way to force the QODBC driver to interpret the input data as UTF-8 and thereby avoid this issue?

I am using Qt 5.5.0 MinGW 32 on Windows.

SGaist

Hi,

Maybe QTextCodec might be of interest.

Hope it helps

mjsurette

Thanks for your answer.

I had considered QTextCodec, but dismissed it. My reasoning was that the corruption, for lack of a better word, happens because the database is in UTF-8 and the QODBC driver seems to read it as UTF-16, so by the time I can get to the damage is done. The solution would be to get QODBC to realize that it needs to read the data as UTF-8 and do the proper conversion to UTF-16 as required by QSting.

My analysis may be wrong of course, so I will try your suggestion and let you know.

mjsurette

Here is a snippit showing what I'm trying to do.

_qbModel= new QSqlTableModel(this,_qbDB);
_qbModel->setTable("Item");
_qbModel->select();
ui->tvParts->setModel(_qbModel);

where ui-.tvParts is a QTableView

Here is one of the fields shown.
General Maintenance and Calibration - FK11 / Maintenance G�n�rale

As you can see, a transformation has already happened and the information has been lost. I really don`t see where I could add a QTextCodec.

hskoglund

Hi, that looks awfully like a QODBC problem I had, when trying to retrieve text data from Visual Foxpro databases with Swedish characters into a QString, A-Z came out fine, but ÅÄÖ came out as �� i.e. same syndrome you're experiencing.

I struggled for a while with QODBC trying various tricks but in the end I caved in and coded my own ODBC-retrieval function: (getting Swedish text into a QString) it's wasn't that hard. If it will help you, you can have a look, its all on Github
(look for the functions singleRowColumnUmlautKludge() and singleRowUmlautKludge())

mjsurette

This looks interesting. Over my head a bit as I lack native Windows experience, but interesting. I`ll have to try decyphering it.

Thanks.

hskoglund

It's a bit undocumented you could say :-)
To push your deciphering effort along I just uploaded a simple statistics program which uses those xxxUmlaut() functions to retrieve Swedish texts. It's compiled with Qt5.4/MSVC2013 but I just tried with Qt5.5.1/MinGW, works fine Sample program on Github