barcode reader with unicode chars

gregorf

Hello,

I have a simple demo with QTextEdit on Win desktop and I'm reading 2D code with Japanase/Chinese characters using barcode scanner. Barcode scanner is configured properly as I can get proper chars in Word or in simple .Net form application. Using native QT, I can append proper characters to QTextEdit from QString (line 1 on the pic). But when reading the 2D code with barcode scanner, latin chars are ok as Japanase/Chinese chars are shown with unicode hex at the front (line 2 on the pic).

What is the way for proper formatting here? Thank you.

#include <QtGui>
#include <QLabel>
#include <QApplication>
#include <QPushButton>
#include <QVBoxLayout>
#include <QTextEdit>
#include <QFontDialog>

int main(int argc, char **argv) {
	
    // base
    QApplication app(argc, argv);

    // window with the title
    QWidget window;
    window.setWindowTitle("Qt-Example");

    // label
    QLabel *label = new QLabel("Hello Qt!");
    label->setAlignment(Qt::AlignCenter);
    
    bool ok;
    QFont font = QFontDialog::getFont(&ok, &window);

    QTextEdit *textEdit = new QTextEdit;
    
    if (ok) {
        textEdit->setFont(font);
    }

    QString str = "Hello ABCabcたくみタクミ匠";
    textEdit->append(str);

    // exit button - ends the application
    QPushButton *button = new QPushButton("&Exit");
    QObject::connect(button, SIGNAL(clicked()), &app, SLOT(quit()));

    // layout the label, text and button
    QVBoxLayout *layout = new QVBoxLayout;
    layout->addWidget(label);
    layout->addWidget(textEdit);
    layout->addWidget(button);
    window.setLayout(layout);

    // show the window and starts the app
    window.show();
    return app.exec();
}

hskoglund

Hi, if you scan the same 2D code into Notepad instead of your app, does Notepad show proper chars like Word or unicode hex in front like in your Qt app?

Christian Ehrlicher

How to you read the data? As simple keyboard input?

Pablo J. Rogina

@gregorf said in barcode reader with unicode chars:

QString str = "Hello ABCabcたくみタクミ匠";
textEdit->append(str);

This code snippet shows that QTextEdit widget is working fine to handle Unicode chars, as the 1st line is displayed properly so it looks like the issue is on how you're handling the input from barcode reader (code snippet that you didn't provide) as the 2nd line shows the hex value for such Unicode chars

gregorf

Barcode scanner (Zebra DS2208) is connected as a HID Keyboard device (USB) and it's configured to support Japanese/Chinese characters. There is no additional code as QTextEdit is getting data as it would from the keyboard.

Reading 2D code into the notepad shows proper results (without hex codes).

Christian Ehrlicher

Then I would say either Qt's key composition is not correct or the keyboard emulation of the scanner is doing something which is not completely correct (I think this is more likely since Qt is only getting it's data from windows)

gregorf

Trying to continue on the task. I implemented a key event filter and there is a difference between the unicode char read by barcode scanner and unicode char entered via keyboard ALT+5320.

Barcode scanner (first line on pic):

QKeyEvent(KeyPress, Key_Alt, AltModifier)
QKeyEvent(KeyPress, Key_Alt, AltModifier)
QKeyEvent(KeyPress, Key_Alt, AltModifier)
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_5, AltModifier, text="5")
QKeyEvent(KeyPress, Key_5, AltModifier, text="5")
QKeyEvent(KeyPress, Key_3, AltModifier, text="3")
QKeyEvent(KeyPress, Key_3, AltModifier, text="3")
QKeyEvent(KeyPress, Key_2, AltModifier, text="2")
QKeyEvent(KeyPress, Key_2, AltModifier, text="2")
QKeyEvent(KeyPress, Key_0, AltModifier, text="0")
QKeyEvent(KeyPress, Key_0, AltModifier, text="0")
QKeyEvent(KeyPress, 0, text="?")
QKeyEvent(KeyPress, 0, text="?")

Keyboard (second line on pic):

QKeyEvent(KeyPress, Key_Alt, AltModifier)
QKeyEvent(KeyPress, Key_Alt, AltModifier)
QKeyEvent(KeyPress, Key_Alt, AltModifier)
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_5, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_5, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_5, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_3, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_3, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_3, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_2, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_2, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_2, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_0, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_0, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, Key_0, AltModifier|KeypadModifier)
QKeyEvent(KeyPress, 0, text="?")
QKeyEvent(KeyPress, 0, text="?")

Diff is in AltModifier|KeypadModifier vs AltModifier|text and I get unicode hex in text with actual char in QT app. Now trying to check if scanner has some useful settings to play with. Still interesting that notepad and other applications handle this properly.

hskoglund

Hi, think this might be a bug in Qt, if you read on Wikipedia you have to enter a registry key EnableHexNumpad in HKCU\Control Panel\Input Method (at least on my english-only Windows 10 PC I had to do it) to be able to enter Unicode characters.

To enter Unicode chars, first you begin by typing Alt and the + key on the numeric keypad. Once you've done that, you can either use the digits on the numeric keypad or you can use the normal digits above the letters. (This is in contrast to the ancient way of entering ascii chars using the Alt-Key, then only the digits on numeric keypad were allowed.)

So your keyboard scanner is correctly set, and Qt needs to adhere to this standard of entering Unicode char.

Christian Ehrlicher

I can add characters the way described in the wikipedia article in a QLineEdit.

gregorf

Yes, I have EnableHexNumpad set in the registry and can enter ALT+hex. As tested in QTextEdit, QPlainTextEdit and QLineEdit I'm getting proper characters. Anyway barcode reader behavior is the same in every tested QT edit controls although key sequence seems similar to manual entry via keyboard.

hskoglund

This bug in Qt is easy to reproduce on any Windows 10 PC:
first enable the Unicode input method described in Wikipedia.

Then create a vanilla Qt Widget app with a single edit control.

If you enter Unicode chars starting with the numpad + key and then only use the digits on the numeric keypad, Qt's edit control behaves fine just like Notepad.

But if you enter Unicode chars starting with the numpad + key and then use the digit keys above the letters on the keyboard, Notepad will handle that with no side effects but Qt will not :-(

By pure accident/chance your barcode scanner is set to use input method #2 and not input method #1, but both should really work the same (as Notepad shows).

gregorf

Great point, thank you. Checking key event sequence confirms your findings that scanner uses method #2. Will review the scanner settings for additional options. Hopefully there is something to override the Qt behavior.

hskoglund

If you don't find any way to change the scanner's behavior, as a workaround for Qt you could try fixing it yourself.

That would mean adding an event handler to your Qt program that checks for keyboard events with ALT+numeric key (but without the KeypadModifier), then removing that key event and instead inserting a new keyboard event with the same ALT+numeric key and with KeypadModifier added to it. I.e. your Qt program would simulate that the scanner uses input method #1:-)

hskoglund

Hi, had a walk this morning when I realized your scanner has no choice but to use input method #2.

Remember that name for the registry entry, EnableHexNumpad? It's called ...hex.. for a reason, i.e. sometimes you also need to use the letters A-F for entering Unicode characters, and those are not available on the numeric keypad (so they will always display as junk in a Qt program).

Just created a Qt bug entry where I used the Unicode Leftwards Arrow U+2190 as the first example, and then Unicode Leftwards White Arrow U+21E6 as an example where you cannot avoid getting junk characters in your Qt program.

gregorf

Hi, thank you for thinking on the matter and for reporting the bug. Yes, method #1 is missing the hex letters. Anyway I did try some scanner settings as there are different keyboard emulations supported, just no real effect as expected.

Entering ALT+21E6 also stops my demo Qt app as E is detected in the sequence :-). ALT+21D6 is ok .. could probably override that ALT+E, just notepad or other apps do nothing during unicode sequence besides char conversion at the end.

I'll keep the manual event handling on the hold for now. I'm planning to check the option where scanner sends actual hex codes with prefix/suffix and i'll try to switch them into actual char manually. Also need to review the scanner and Qt behavior on Linux. Will post some findings.

Christian Ehrlicher

@hskoglund But Qt Quick and Widgets team? It's a core input problem and has nothing to do with widgets :)

hskoglund

@Christian-Ehrlicher Sorry my bad :-(
Indeed in this case the widgets are innocent for once.
P.S. Changed the title of the bug entry to avoid blaming the widgets.

BTW, do you know anything about that new widget on/off switch that was promised I think late 2018?

Christian Ehrlicher

@hskoglund said in barcode reader with unicode chars:

that new widget on/off switch

Never heard of, any references?

hskoglund

@Christian-Ehrlicher https://www.qt.io/blog/2019/02/22/qt-roadmap-2019 search for "switch"
Granted, there is one in QML https://doc.qt.io/qt-5/qml-qtquick-controls2-switch.html but I really like to work with widgets (I even use them in my iPhone Qt app)

gregorf

Hi, I'm playing with the key press event filters to manually convert the hex code and found some more details for the hexcode + char display combination (method #2) which might be useful for the fix.

Keypress event comes from several objects and somehow Alt+ combination seems to work for one of them while the second displays the hex code:

QKeyEvent(KeyPress, Key_Alt, AltModifier) QWidgetWindow(0x1f4300f0, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_Alt, AltModifier) QTextEdit(0x1f416a50)
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier) QWidgetWindow(0x1f4300f0, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier) QTextEdit(0x1f416a50)
QKeyEvent(KeyPress, Key_5, AltModifier, text="5") QWidgetWindow(0x1f4300f0, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_5, AltModifier, text="5") QTextEdit(0x1f416a50)
QKeyEvent(KeyPress, Key_3, AltModifier, text="3") QWidgetWindow(0x1f4300f0, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_3, AltModifier, text="3") QTextEdit(0x1f416a50)
QKeyEvent(KeyPress, Key_2, AltModifier, text="2") QWidgetWindow(0x1f4300f0, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_2, AltModifier, text="2") QTextEdit(0x1f416a50)
QKeyEvent(KeyPress, Key_0, AltModifier, text="0") QWidgetWindow(0x1f4300f0, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_0, AltModifier, text="0") QTextEdit(0x1f416a50)
QKeyEvent(KeyPress, 0, text="?") QWidgetWindow(0x1f4300f0, name = "WidgetClassWindow")
QKeyEvent(KeyPress, 0, text="?") QTextEdit(0x1f416a50)

Catching the ALT, + and 0-F keys for one of the objects (QTextEdit or QWidgetWindow) and not forwarding them further, actually makes the application work and result is only the real character displayed.

Final result where mentioned key events for QTextEdit are filtered out:

QKeyEvent(KeyPress, Key_Alt, AltModifier) QWidgetWindow(0x1d5f0510, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_Plus, AltModifier|KeypadModifier) QWidgetWindow(0x1d5f0510, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_5, AltModifier, text="5") QWidgetWindow(0x1d5f0510, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_3, AltModifier, text="3") QWidgetWindow(0x1d5f0510, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_2, AltModifier, text="2") QWidgetWindow(0x1d5f0510, name = "WidgetClassWindow")
QKeyEvent(KeyPress, Key_0, AltModifier, text="0") QWidgetWindow(0x1d5f0510, name = "WidgetClassWindow")
QKeyEvent(KeyPress, 0, text="?") QWidgetWindow(0x1d5f0510, name = "WidgetClassWindow")