How is QKeyEvent supposed to be used for games?

[ThomH]

As a quick experiment I tried the following sequence of keypresses on macOS:

• press the key labelled with . and >;
• press shift;
• release the ./> key;
• release shift.

Via QKeyEvent::key() I received:

• an initial Key_Period press event, followed by a bunch more (as key repeat);
• a Key_Shift press event;
• a Key_Greater press event (followed by key repeats);
• a Key_Greater release event; and
• a Key_Shift release event.

The most obvious obstacles with trying to use key() for a game:

• at no point was a Key_Period release received — per the reported events, that key is pressed indefinitely; and
• conflating the two keypresses to produce Key_Greater is obviously unhelpful.

The documentation notes that nativeScanCode isn't usable on the Mac, so I skipped that.

Via ::nativeVirtualKey I received the same sequence of messages, obviously, but with a separate flaw:

• I received a press for kVK_ANSI_Period (in old HIToolbox terms);
• I then get a press with a nativeVirtualKey of 0 when I also press shift. So for some reason Qt blocks macOS's built-in value for shift of 0x38 (kVK_Shift) and thereby blocks the built-in value for kVK_ANSI_A, which actually is 0;
• these are followed with appropriately-timed release events for the same virtual keys.

So the most obvious obstacles with trying to use nativeScanCode for a game:

• Qt declines to pass on the scan codes for anything it considers a modifier; and
• as a result it makes kVK_ANSI_A ambiguous. I cannot safely support the 'a' key.

In an ideal world I'd be able to use ::key() but tell Qt not to apply modifiers. I assume that's not an option so what is the intended use of QKeyEvent?

[johngod]

I'm not on my mac right now, but I tryed on windows and it seems to work. If I understood you correctly you're pressing 3 keys: "." "<" and "shift" then releasing them one at the time. I can catch the all 3 press and release events.

Either way if you're trying to move a person in your game while keeping a key pressed with auto repeat, while catching the key press events, probably the result will be very ugly.
AFAIK there's not much consistency in the velocity the key press is fired and if your pressing, for example, the arrow up key for move up then alternate between left and right keys things can get messy.
This is how I do it: I usually disable auto repeat and catch only key press and realese events. I create a bool variable for each key I want to use, something like isKeyUp_pressed, wich are set and unset with the key press and key release events, you get the picture. Whenever some key is pressed I run a timer at my desired frame per second velocity. While the timer is running I just check wich keys are pressed and act upon it. There's a great advantage in using a timer because you can control the speed of the actions to take for the keys pressed and that gives you a very smooth movement of your charaters.
I usually also create a signal for each key. If in qml then I can create my own keyboard component with convenient signals for each key, then I can connect thoose signals to command whatever actions I want in the game.
Hope this helps.

[ThomH]

No, to explain again:

There is a single key on my keyboard labelled with both '.' and '>'. You will see this key on, at least, the standard layouts for the US and every European keyboard that I'm familiar with. I'll just call it the '.' key.

My sequence of presses is:

• press the '.' key;
• press the shift key;
• release the '.' key;
• release the shift key.

I don't care about key repeat, it isn't interesting to me.

From that sequence I see:

• I cannot use QKeyEvent::key() to track which keys are up and which are down because — again, ignoring key repeat — it reports three key presses, for Key_Period, then Key_Shift, then Key_Greater but only two key releases, for Key_Greater then Key_Shift. It never reports a key release for Key_Period;
• I cannot use QKeyEvent::nativeScanCode because the documentation is explicit that it isn't supported on macOS; and
• I cannot use QKeyEvent::nativeVirtualKey because the value for the 'a' key and the sentinel value of 0 are identical, so I can't actually sense every key in the standard alphabet, including one that is in the default mapping for almost every modern game, and it doesn't report native virtual key values for modifiers, so I also can't use shift, control, alt, etc.

The question is purely about Qt seemingly being not fit for purpose since there is no reliable way to track key states.

This piece of software is already fully functional under other toolkits, this isn't a logic question.

[johngod]

@ThomH Now I understand what you mean and I can confirm that behaviour in my mac. (Btw that key in portuguese layout is "." and ":" ). The only solution I can think of is for you to emit a "false" release event for Key_Period when Key_Greater is realeased. This may be a bit of work if you have other keys combinations with shift, that you want to catch.

[ThomH]

@johngod Yeah, it seems to be all of the keys other than alphabetics that do three pressed but only two releases if you press shift second.

To be even more clear for other readers, with a very basic application I added the following to my QMainWindow subclass:

void MainWindow::keyPressEvent(QKeyEvent *event) {
    if(event->isAutoRepeat()) return;
    qDebug() << "Press" << event->key() << event->nativeVirtualKey() << event->modifiers() << event->nativeScanCode();
}

void MainWindow::keyReleaseEvent(QKeyEvent *event) {
    if(event->isAutoRepeat()) return;
    qDebug() << "Release" << event->key() << event->nativeVirtualKey() << event->modifiers() << event->nativeScanCode();
}

I then tested the sequence as above under both macOS — press the key labelled both . and > on US keyboards, press shift, release ., release shift — and Ubuntu 19.04.

Observing that isAutoRepeat() is unreliable, output on Ubuntu is:

Press 46 46 QFlags<Qt::KeyboardModifier>(NoModifier) 60
Release 46 46 QFlags<Qt::KeyboardModifier>(NoModifier) 60
Press 46 46 QFlags<Qt::KeyboardModifier>(NoModifier) 60
Release 46 46 QFlags<Qt::KeyboardModifier>(NoModifier) 60
Press 46 46 QFlags<Qt::KeyboardModifier>(NoModifier) 60
Release 46 46 QFlags<Qt::KeyboardModifier>(NoModifier) 60
Press 46 46 QFlags<Qt::KeyboardModifier>(NoModifier) 60
Press 16777248 65506 QFlags<Qt::KeyboardModifier>(ShiftModifier) 62
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Press 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Press 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Press 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Press 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Press 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Press 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 16777248 65506 QFlags<Qt::KeyboardModifier>(NoModifier) 62

If I remove most of the ones that obviously are unflagged autorepeat then:

Press 46 46 QFlags<Qt::KeyboardModifier>(NoModifier) 60
Press 16777248 65506 QFlags<Qt::KeyboardModifier>(ShiftModifier) 62
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Press 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 62 62 QFlags<Qt::KeyboardModifier>(ShiftModifier) 60
Release 16777248 65506 QFlags<Qt::KeyboardModifier>(NoModifier) 62

So, no release for key() 46, ever. But I do get a release for 62 before I get a press. You'd think maybe I can use the nativeScanCode() to dig myself out of that hole, but testing the same code and key sequence on the same computer under macOS 10.15.5 produces:

Press 46 47 QFlags<Qt::KeyboardModifier>(NoModifier) 1
Press 16777248 0 QFlags<Qt::KeyboardModifier>(ShiftModifier) 0
Release 62 47 QFlags<Qt::KeyboardModifier>(ShiftModifier) 1
Release 16777248 0 QFlags<Qt::KeyboardModifier>(NoModifier) 0

In this case isAutoRepeat() is possibly trustworthy but nativeScanCode() explicitly isn't supported under macOS per the documentation (and, in practice, seems to return only either 1 or 0).

So on both platforms I get no release for key() 46 ever, and the only empirically-suggested workaround seem to be: if X11, match by the nativeScanCode field; but if macOS then match by the nativeVirtualKey() field. I don't think that's sustainable, besides anything else because I don't have time to test every OS Qt supports and try to figure out if/how to undo its wacky failure to report key releases on each.

I think @johngod's suggestion — that I have to make assumptions about which keys are really the same keys — is probably most sustainable since it relies on locale+platform keyboard layout, which is heavily nuanced but at least a fact.

Either that or pull in something like SDL just for its keyboard API (!)

If anybody could answer as to what the Qt engineers intended here that might help to inform a better strategy for dealing with this.

[johngod]

You can try asking on qt interest mailing list, I think there are more developers over there that would answer better about their plans.
Any way, you din't mention what's your use case, I guessing you want to do something like this:

if (Key_Greater is release) 
    doKeyGreaterReleaseStuff();
if (Key_Period is release) 
    doKeyPeriodReleaseStuff(); //doenst work

Assuming that KeyPeriod release should be fired along with keyGreater released, may you can get way with the following logic:

if (Key_Greater is release)  {
    doKeyGreaterReleaseStuff();
    doKeyPeriodReleaseStuff(); 
}

About locale I'm not sure if you need to worry about that, I'm on portuguese keyboard and got the same 46 code for that "key period", even if my the symbols are diferent.

Cheers

[nyanpasu64]

I've been having a similar issue, since I'm developing a music program that treats the computer keyboard as a piano-shaped layout for note entry. I decided to do some research on how other Qt-based programs get physical keyboard layout.

QtWebEngine and Chromium

If I understand correctly, QtWebEngine translates QKeyEvent to platform-independent JavaScript KeyboardEvent.code (MDN docs). I tried visiting the W3C "Keyboard Event Viewer" web app, and it works fine in the QtWebEngine-based Falkon browser on Linux Wayland (only tested QWERTY) and Xorg (QWERTY and Dvorak layouts).

If this functionality is fully working, I think I could extract this functionality into a reusable library, to obtain platform-independent physical keycodes from Qt, using DOM names for these keycodes. There's certainly demand, as this thread, another qt.io thread, and several broken Qt programs (pressing 1 then Shift causes a stuck key) demonstrate.

My concern is, if the ambiguity between kVK_ANSI_A and modifiers is truly unavoidable on Mac, then QtWebEngine will have KeyboardEvent/etc. input errors on Mac, and so will my key conversion library. Has anyone tested to see if QtWebEngine has key input issues? I don't have a Mac to test with.

QuteBrowser seems like an active QtWebEngine-based browser available on Mac. Falkon isn't available on Mac. Otter Browser's binary packages look awfully unmaintained, and I couldn't find a .dmg download on SourceForge.

I haven't found any Mac-specific issues on the bug tracker searching for https://bugreports.qt.io/secure/QuickSearch.jspa?searchString=KeyboardEvent, nor searching for QtWebEngine key or QtWebEngine keyboard.

Deep dive

Qt's nativeKeyCodeForKeyEvent() translates QKeyEvent into platform-specific keycodes understood by Chromium. The function contains the comment, "Ifdefs here should match <ui/events/keycodes/dom/keycode_converter.cc>, since that is where the native key code is eventually used." I think this function could be improved to mention the specific function NativeKeycodeToDomCode().

Qt's WebEventFactory::toWebKeyboardEvent(QKeyEvent *ev) calls nativeKeyCodeForKeyEvent(), stores it into webKitEvent.native_key_code, and passes it into Chromium's NativeKeycodeToDomCode().

Chromium's DomCode NativeKeycodeToDomCode(uint32_t usb_keycode) has the doc comment "Convert a native (Mac/Win/Linux) keycode into a DomCode." The implementation is fairly simple:

DomCode KeycodeConverter::NativeKeycodeToDomCode(int native_keycode) {
  for (auto& mapping : kDomCodeMappings) {
    if (mapping.native_keycode == native_keycode)
      return static_cast<DomCode>(mapping.usb_keycode);
  }
  return DomCode::NONE;
}

The file dom_code_data.inc is a massive table much like an X-macro. This file allows the user to construct an arbitrary mapping between different representations of keycodes. It repeatedly invokes the DOM_CODE macro on USB scancodes (also used for DomCode enum values), evdev/XKB/Windows/Mac keycodes (I think XKB is always evdev+8), nullable char const * string literals, and C identifiers.

Chromium's DomCode is an enum with keycode names and USB scancode values, populated using dom_code_data.inc. Chromium uses USB scancode values to uniquely identify/represent hardware keys. (Wouldn't it be a lot simpler if Qt gave you a hardware key directly? 😉 Sadly OSes don't give you this information.)

Chromium's kDomCodeMappings is an array of KeycodeMapEntry, also populated using dom_code_data.inc.

Chromium's KeycodeMapEntry is a struct containing a USB keycode, native keycode (platform-dependent), and KeyboardEvent.code name.

The platform-dependent values (sent by Qt) are found in the KeycodeMapEntry::native_keycode field, with the following description:

  // Contains one of the following:
  //  On Linux: XKB scancode
  //  On Windows: Windows OEM scancode
  //  On Mac: Mac keycode
  //  On Fuchsia: 16-bit Code from the USB Keyboard Usage Page.
  int native_keycode;

The JS DOM key names are found in the code field, with the following description:

  // The UIEvents (aka: DOM4Events) |code| value as defined in:
  // http://www.w3.org/TR/DOM-Level-3-Events-code/
  const char* code;

How do https://www.w3.org/TR/DOM-Level-3-Events-code/#code-value-tables and https://www.w3.org/TR/uievents/#keys-codevalues and https://www.w3.org/TR/uievents-code/ compare? I haven't looked through them yet.

I think DomCodeToCodeString() converts DomCode to KeyboardEvent.code, but I haven't verified.

If I were to write my own API, I would have to choose between exposing the DomCode enum to users (fast comparison), or a CodeString string-based API (introspecting for key names), or both.

---

Back to Qt... Qt's WebEventFactory::toWebKeyboardEvent(QKeyEvent *ev) calls NativeKeycodeToDomCode(), and assigns the return to webKitEvent.dom_code. The function eventually returns webKitEvent.

Qt's content::NativeWebKeyboardEvent WebEventFactory::toWebKeyboardEvent(QKeyEvent *ev) is called in exactly one place where Qt forwards key events to Chromium. toWebKeyboardEvent()'s return value is stored as webEvent and passed to m_rwhv->host()->ForwardKeyboardEventWithCommands(webEvent, latency, &commands, nullptr);. I don't understand this, but it's not relevant to how QtWebEngine converts QKeyEvent to platform-independent keycodes.

---

webKitEvent is of Chromium's type content::NativeWebKeyboardEvent inheriting from WebKeyboardEvent.

(Chromium's WebKeyboardEvent) has the following interesting comment:

  // The actual key code genenerated by the platform. The DOM spec runs
  // on Windows-equivalent codes (thus |windows_key_code| above) but it
  // doesn't hurt to have this one around.
  int native_key_code = 0;

I theorize windows_key_code maps to KeyboardEvent.keyCode (deprecated), dom_code maps to KeyboardEvent.code (physical key location), and dom_key maps to KeyboardEvent.key (locale-specific and affected by shift). Of course, for our purposes, we only care about dom_code.

Is that correct? https://source.chromium.org/chromium/chromium/src/+/master:third_party/blink/renderer/core/events/keyboard_event.cc;l=130-134;drc=052831f0220b79fe0c3343b49f6d2863ea6de05d It's complicated and I don't fully understand.

Unresolved questions

• Does QtWebEngine actually use QKeyEvent? I'd have to build it from scratch and stub out the functions to find out.

Other apps

Prior to figuring out QtWebEngine, I also explored other Qt apps with special handling of QKeyEvent.nativeScanCode() or nativeVirtualKey.

OpenClonk

OpenClonk (a game with a Qt-based GUI) has an interesting approach to handling keycodes, subtracting 8 from Linux XKB keycodes and doing something with Mac keycodes. I wonder if the game's keyboard input is unusable on Mac, since it returns event.nativeScanCode(); which is useless on Mac. https://github.com/openclonk/openclonk/blob/2bb4b62f0b8940812c4fd5269288813790a9102b/src/editor/C4ConsoleQtViewport.cpp#L313

I get the feeling that QKeyEvent is only used for the game's editor. I think the actual game uses a different keyboard input with platform dependencies: https://github.com/openclonk/openclonk/blob/master/src/gui/C4KeyboardInput.cpp

In the end, I decided to use QtWebEngine's key handling code as a path towards platform-independent key events.

QtWebKit

QtWebKit is no longer developed in Qt. I'm not sure if there are any browsers using https://github.com/qtwebkit/qtwebkit, and I haven't looked into how it extracts platform-independent position-based keycodes/scancodes. I can't search this repo because it's a clone.

So I spent the time to shallow-clone to disk and rg through the files. Source/WebKit/UIProcess/API/wpe/qt/WPEQtViewBackend.cpp might be relevant, but Source/WebCore/platform/qt/PlatformKeyboardEventQt.cpp looks disappointing once I actually open the file.

Dolphin Emulator

Dolphin uses platform-specific APIs for keyboard/mouse/gamepad input. That might be a workable solution, but I don't have a Mac and don't know how well this will mesh with Qt widget event propagation. And I don't need gamepad or background input (but you might).

[nyanpasu64]

I released the library: https://github.com/nyanpasu64/qkeycode/

Qt declines to pass on the scan codes for anything it considers a modifier; and
as a result it makes kVK_ANSI_A ambiguous. I cannot safely support the 'a' key.

I worked around this problem by treating nativeVirtualKey()=0 as "A" when nativeScanCode() is 1, and ignoring it otherwise. Is that a good solution, or will it run into other issues?

[ThomH]

Fantastic work! I'll need to do some research re: licensing but hopefully you've solved my problem!

Otherwise, I tested quitebrowser under macOS with the W3C keyboard event viewer. Pressing and releasing 'a' seemed to work correctly; reported events for the 'legacy' portion of the table were:

#	Event type	charCode	keyCode	which
5	keyup	0	65 a	65
4	input	-	-	-
3	beforeinput	-	-	-
2	keypress	97 a	97	97
1	keydown	0	65 a	65

[nyanpasu64]

Otherwise, I tested quitebrowser under macOS with the W3C keyboard event viewer. Pressing and releasing 'a' seemed to work correctly; reported events for the 'legacy' portion of the table were:

When pressing the QWERY a key (right of caps lock), QtWebEngine synthesizes DOM code from QKeyEvent::key(). Which means that switching keyboard layouts causes the key to produce incorrect or missing code. I reported this at https://bugreports.qt.io/browse/QTBUG-85660, and my library has a workaround for this issue.

Incidentally I found that Qt Creator's QtWebEngine sample browser is easier to use than qutebrowser, and sends keys directly to the site instead of capturing them unless you're in the right mode.