Adding unknown keysym until keysym table is full and no further input possible

[merkata]

I have a (Arch Linux) client box connecting via vncviewer to a (Arch Linux) server running (tigervnc) vncserver. On my client I use two languages (setxkbmap -layout us,bg (phonetic) -option grp:alt_shift_toggle) - switching and using both languages on the client is fine, both languages work on the server when in a terminal session, however when I connect to the VNC desktop, keys stop responding after a while. Relevant errors in ~/.vnc/hostname:1.log

...
Thu Dec 18 11:32:13 2014
Input:       Added unknown keysym 0x69 to keycode 221
Input:       Added unknown keysym 0x66 to keycode 219

Thu Dec 18 11:35:04 2014
Input:       Added unknown keysym 0x78 to keycode 217

Thu Dec 18 11:35:05 2014
Input:       Added unknown keysym 0x62 to keycode 202

Thu Dec 18 11:35:19 2014
Input:       Added unknown keysym 0x75 to keycode 197
...

Running xev shows some weird stuff - when pressing 'd' for instance

KeyRelease event, serial 38, synthetic NO, window 0x1200001,
    root 0x178, subw 0x1200002, time 175414596, (49,39), root:(50,64),
    state 0x2000, keycode 149 (keysym 0x64, d), same_screen YES,
    XKeysymToKeycode returns keycode: 40
    XLookupString gives 1 bytes: (64) "d"
    XFilterEvent returns: False

and xmodmap -pke would report that 'd' was added to 149

xmodmap -pke | grep 149
keycode 149 = d D d D

The table has room for 255 entries and they fill up really quickly - when it is full, I'm not able to use any 'unknown' keys that werent added to the table before it was full.

Locales on both machines

bg_BG.utf8
C
en_US.utf8
POSIX

tigervnc is at latest version (1.4.0-1 from official arch repository)

[CendioOssman]

This is an unfortunate limitation in the current architecture. So I'm afraid we don't have any easy fix for it. As a workaround you can change the keyboard layout in the VNC session to reset things.

[kozhin]

The same problem I have with CentOS 7. My locale is ru_RU.utf8.
Is there any way to reset the table when it's full?

[CendioOssman]

Yes. Reloading a fresh keymap with setxkbmap will reset the table.

[kozhin]

I mean how can I modify the vncAddKeysym() function call to reset the table. I'm not so good at C coding but I understand that the reset can be possibly done here or inside the function. But I don't know how exactly to reset the table (table array name and method to reset). Can you suggest please?

[CendioOssman]

Not sure. It's not trivial as we cannot easily tell what we've added and what has been added by the user.

[kozhin]

I've spent some time to investigate the issue and found that everything works fine if the local language is correspondent to the input layout language of a client.
For example, if server's keyboard layout is "EN" when I type on my iPad using Russian keyboard layout the problem happens.
But if I change server's keyboard layout to "RU" when I type again using Russian keyboard layout everything works fine.
Also I've noticed the difference in logs. In the first case I see "Added unknown keysum" message each time I press Russian symbols. In the second case nothing appears in logs.

At the same time if I use my Mac's VNC client, I can't use Russian characters in any of the cases. In the first case I see in logs "Added unknown keysum" but I don't see any characters appear. This happens until the table is full.
In the second case I see "Failture adding new keysum" when I type.

It seems that some VNC clients handles with this issue in some way, some doesn't do anything.

Hope this will help.

[romanrm]

So, should we just run setxkbmap every minute in crontab, or...?
This is unusable as is, either some cyrillc or some latin symbols will stop working randomly after a while.

[CendioOssman]

Setting the same layout as you have on the client should avoid most issues. Swapping between very different layouts will most likely require you to swap in the VNC session as well though.

You can also try the new RawKeyboard setting that's available in the development version. It does not suffer from this limitation.

[romanrm]

Setting the same layout as you have on the client should avoid most issues. Swapping between very different layouts will most likely require you to swap in the VNC session as well though.

We swap between "us" and "ru" layouts all the time, a couple of times per minute is typical. In my case the client is Windows 7, two keyboard layouts are installed there, switched by Ctrl+Shift. What should I set in the VNC session?

Tried at first the usual "physical X server style" invocation,

setxkbmap -layout us,ru\(winkeys\) -option -option grp:ctrl_shift_toggle

then, simplified,

setxkbmap -layout ru -option

Neither helps completely. After a few minutes of casual input, some keys will randomly stop being accepted. Message in the log is as mentioned above, "Failure adding new keysym..."

The only "solution" was to run setxkbmap again. Then it logs "Added unknown keysym" and inputs fine. And seriously, setting up the latter variant in crontab (while ensuring the user and DISPLAY variables are correct) to run every 5 minutes seemed enough to not let the issue reoccur.

This problem does not happen with the "stock" VNC server from Debian 8. In fact that one does not support the XKB extension at all. But us/ru input still works there, and does not fail over days or even months of operation.

[CendioOssman]

Tried at first the usual "physical X server style" invocation,

setxkbmap -layout us,ru(winkeys) -option -option grp:ctrl_shift_toggle

This will generally not work well since there is no mechanism to sync the switching between the layouts.

then, simplified,

setxkbmap -layout ru -option

This should work as long as you are using the Russian layout on the client. If you switch to American then you'll have to call setxkbmap again with us as the argument.

Are you saying this case isn't working?

This problem does not happen with the "stock" VNC server from Debian 8. In fact that one does not support the XKB extension at all. But us/ru input still works there, and does not fail over days or even months of operation.

It should have the same behaviour. Can you check exactly what that server is?

[romanrm]

This should work as long as you are using the Russian layout on the client. If you switch to American then you'll have to call setxkbmap again with us as the argument.

-layout ru results in Cyrillic symbols entered being accepted quietly; but Latin symbols result in the "Added unknown keysym" message in the logs. I guess this is expected up to this point. What is not expected, is that after a few minutes of operating like that, some Latin characters stop being accepted (no pattern which ones), and instead start to produce the same "Failed to add unknown keysym" message.

If you switch to American then you'll have to call setxkbmap again with us as the argument.

How would I do that? As I said we switch multiple times per minute casually. The keyboard layouts are changed on the client via a simple press of Ctrl+Shift, the VNC client is just another regular application there, it gets whatever characters currently selected layout's keypresses result in. So practically it gets an intermixed stream of Cyrillic + Latin characters. I don't think there's a mechanism to notify the server of a layout change on the client and then execute a specified program there.

RawKeyboard that you mentioned sounds promising, but I did not try that yet.

It should have the same behaviour. Can you check exactly what that server is?

I meant not TigerVNC in Debian, but the original VNC server in Debian Jessie (pre-TigerVNC migration), this one: https://packages.debian.org/jessie/vnc4server
With it,

$ setxkbmap -layout ru -option
XKB extension not present on :1.0

But no problems with input whatsoever.

[kmatveyonok]

I set keyboard switch to application on client (KDE, Krdc). Some time it switch layout on server with Ctrl+Shift synchronously wich client some time it loose switch key press and client and server becomes in different layouts and this problem happens.
As romanrm wrote, switches happens many times per minute and this is some kind of annoying behaviour.

[CendioOssman]

If you switch to American then you'll have to call setxkbmap again with us as the argument.

How would I do that? As I said we switch multiple times per minute casually. The keyboard layouts are changed on the client via a simple press of Ctrl+Shift, the VNC client is just another regular application there, it gets whatever characters currently selected layout's keypresses result in. So practically it gets an intermixed stream of Cyrillic + Latin characters. I don't think there's a mechanism to notify the server of a layout change on the client and then execute a specified program there.

By hand unfortunately. Or if you can get a keyboard layout switcher in the desktop tray in the VNC session. This is a use case where we don't really have a good solution at the moment. :/

I meant not TigerVNC in Debian, but the original VNC server in Debian Jessie (pre-TigerVNC migration), this one: https://packages.debian.org/jessie/vnc4server

I assumed as much. But that is RealVNC, which AFAIK uses the same method as TigerVNC. So it's worth investigating why the problem isn't seen with that server.

[CendioOssman]

Alright, so I had a look at vnc4server in Debian 8 and it does indeed have the exact same behaviour as TigerVNC. However it has absolutely ancient keymaps (e.g. no multimedia keys) resulting in lots of free space for extra characters. On a default startup it has 148 available slots. TigerVNC with a current Xorg has 24.

So if you can manage to load such an older layout then you could also avoid the issue.

[romanrm]

So if you can manage to load such an older layout then you could also avoid the issue.

Could you give a hint how to do that? I looked and tightvnc-standalone-server doesn't seem to ship any keymap files. And doesn't seem likely it uses those from xkb-data either.

Also I wonder if I even need such an extensive keymap. I do not use multimedia keys inside the VNC session for instance, I do not watch video or play music over VNC. How would I go about creating or loading a very minimal layout to free up more slots?

[CendioOssman]

Sorry, but I don't know. I have not really played around with the definition files. If you dump things using xkbcomp then you might get something you can you can modify and clean up and then load back in to the server.

[ogurets]

Hello, guys!
Here is a crude solution to our little 8-bit problem: ogurets@441b7bc

Already built a Xvnc from this and it works!
Upd. 3 march 2019: still works and no side-effects noticed. Dear bounty, I'm coming for you!

[sergey1369]

Hello, guys!
Here is a crude solution to our little 8-bit problem: [ogurets/tigervnc@441b7bc]

Hi, Ogurets!

Thanks for your efforts!! Your patch is lifesaver!! :)

I've tried your solution, but it thas troubles with quick input.
When I type slowly it works perfectly, but when I type quickly it
just repeats last key. So I get something like "baccc" for "basic".

Can you think about some enhancements to your method?
For example specify two keyboard maps in .vncrc and quickly
switch them as needed.

UPDATE.
I've found simple workaround. No patches needed!
I put native letters on column 3&4 of us xkb file and load it.

$ setxkbmap vnc_us_ru

VNC works great now. Here is my changes
vnc_us_ru.zip

name[Group1]= "English/Russian";
key <TLDE> { [ grave, asciitilde, Cyrillic_io, Cyrillic_IO ] };
.....

[ghost]

vnc_us_ru map works like a charm. Thank you, @sergey1369

[ghost]

Actually, there's a problem with vnc_us_ru. If you're typing fast, it can be that despite the ru layout some letters remain latin.

[CendioOssman]

That sounds like a generic problem, and not an issue with the keymap. Please open a new issue with more details on your setup. Also please try to include a debug log from the server and/or client.

[ghost]

Never mind. It was only one old app that was affected, one running under Wine at that.

[gujjwal00]

@CendioOssman I have a proposal to fix this. Basic idea is to "remember" the KeyCodes for which KeySyms has been added by TigerVNC, and reuse those when unused keys are no longer available.

1. Each key customized by TigerVNC is assigned a unique name (xkb->names->keys[key].name). The name must not already be in use, and should be unlikely to collide with common key names. For example: TGxx where xx is from 00 to 99.
2. For each customized key, it's Name, KeyCode, and KeySym is tracked in an array local to InputXKB.c, say CustomKeys.
3. If free key is not available, CustomKeys array is checked. If a key has same Name & KeySym in CustomKeys and xkb, It is assumed to be assigned by TigerVNC, and is available for reuse.

With this scheme, for a user/system specified key to be overwritten by TigerVNC, it has to have the same customized Name & KeySym as assigned by TigerVNC, and has to be specified after TigerVNC has assigned it.

To reduce the number of keymap updates, CustomKeys could be reused in least-recently-used order.

I have done some preliminary testing, and wanted to know if this is acceptable and worth implementing.

[CendioOssman]

That sounds like a very reasonable approach, yes. Something similar was done for x0vncserver in #1510.

Ideally, we start sharing more of the keyboard handling between x0vncserver and Xvnc, as they are trying to solve the same problem. We've already done so for RandR handling in unix/common/randr.cxx.

[gujjwal00]

I looked at RandR implementation, and as I understand it, common functionality is implemented in unix/common/randr.cxx while Xvnc/x0vncserver specific details are abstracted via RandrGlue.h.

In case of custom KeySym assignment, difference between Xvnc and x0vncserver seems to be how they retrieve keymap (GetMaster vs XkbGetMap), and announce changed keymap ( XkbSendNotification vs XkbChangeMap). Rest of the keymap modification is almost exactly same.

This common functionality can be moved to something like InputCommon.h & InputCommon.c in unix/common. Existing unixcommon.h would have been a good choice but its a C++ header. By only extracting common functionality, "InputGlue" is not needed for now, but it can added if required.

[gujjwal00]

One issue I found is which XKB headers to use in common code. X11/extension/XKBstr.h is used by x0vncserver, while Xvnc uses xkbstr.h from xserver sources. Both seems compatible to me, but I don't know if this is just a coincidence.

If the header is abstracted via "InputGlue.h", we have to duplicate a lot, essentially nullifying the benifit of shared code.

[CendioOssman]

They are seemingly very similar, but not identical. Which does indeed complicate things...

[gujjwal00]

One option is to treat common files as "source library" instead of "binary library". Both Xvnc & x0vncserver could compile common files separately, and the common files can include appropriate header based on some #define.

This is already happening in some sense via RandrGlue.c

[CendioOssman]

Not ideal given there are different build systems for Xvnc vs x0vncserver. But give it a go, and we can see what it looks like.

[twaik]

I was experiencing simillar problem with binding keysyms to keycodes and wrote solution like this: source (XCB code, not for X server).

Shortly:
Function scans for needed keysym in all available language layouts.
If it finds a keycode it sets needed layout ad sends KeyPress and KeyRelease for all needed modifier keys (Ctrl, Shift, Alt and Meta), sends needed keycode KeyPress and KeyRelease and sends KeyPresses and KeyReleases to reset modifier keys to their previous state (before invoking function).
If it does not find a keycode it gets scratch keycode (see fakeKeys in unix/xserver/hw/vnc/InputXKB.c) and binds it to the needed keysym, sends KeyPress and KeyRelease for needed keycode and sets 200 ms timer to reset given keycode to NoSymbol (to avoid filling table). Also it increments counter used to get different scratch key. Timer and using different scratch codes are needed because some applications do not work properly if you send change-keyboard-mapping event right after sending KeyPress/KeyRelease, I do not know why.

My solution works like a charm in client side program but I am not sure if I can port it to server side. I hope it will help you fix this issue.

P.S. After receiving XCB_XKB_NEW_KEYBOARD_NOTIFY my code is processing keymap to get sorted table of unicodes with corresponding keycodes, groups (layout numbers) and modifiers for binary search. It is needed to type needed unicode symbol as fast as possible using as few resources as possible. This is especially relevant for the case if you are typing on Android device in language that is not present on target PC or if alphabet of this language can not be bound to keyboard layout (i.e. Chinese, Japanese, etc.).

Thank you in advance.

[twaik]

Actually replacing code in InputXKB.c

	for (key = xkb->max_key_code; key >= xkb->min_key_code; key--) {
		if (XkbKeyNumGroups(xkb, key) == 0)
			break;
	}

with

	static int curFakeKeyIdx = 0;
	key = fakeKeys[curFakeKeyIdx++];
	if (curFakeKeyIdx >= ARRAY_SIZE(fakeKeys))
		curFakeKeyIdx = 0;

works even without resetting keys back to scratch.

[McMuppet]

Is this considered fixed then?

[twaik]

No

[McMuppet]

@twaik On May 22 you mention replacing code in InputXKB.c that works without a reset. What did that code help with in this bug's context? Also I searched the repository for that .c file you mentioned and couldn't find it. Is that not in TigerVNC repository?

[twaik]

Do you mean that? It searches scratch keys on the current layout.

[CendioOssman]

This has now been fixed in #1734! :)

[romanrm]

So glad to hear. But I just downloaded the latest release to try out, turns out the release is from Jul 23 with the fix not included yet. :( Thanks