Virtual keyboards, also called on-screen keyboards (OSKs), is a piece of software that allows a user to input keys via a touchscreen or a mouse interface, instead of a physical keyboard.
Broadly speaking, OSKs can be split into two categories:
- Mobile keyboards, primarily for portrait screens, more focussed on simple text input, like what would be found on Android or iOS as default.
- Desktop keyboards, primarily for landscape screens or larger screens, more focussed on controlling a computer to it's fullest extent, including use of modifier keys and keybinds.
Note that these categories are flexible, and just mainly serve as an example of categorisation of OSKs. Others that could exist are simple/advanced and touchscreen/mouse, as examples.
In this document I'll outline what each are typically used for, some notes on what Wayland currently has in place for implementations, some requirements for both protocol + compositor-side and application-side developers (application-side devs here being the ones working on OSKs on Wayland), and finally some examples of those keyboards (mainly Wayland, some X).
- OSKs aren't just primarily for touchscreens, they should be able to be used effectively with mice and other pointer methods as well. This means that some aspects of an OSK's design need to be examined from multiple angles, such as the amount of screen estate that the OSK takes up, as touchscreen OSKs should take up enough for the fingers of user to be able to accurately use the OSK, but mouse users have typically higher precision, so they may prefer a smaller amount of screen estate to be taken up
Mobile keyboards are typically used to input text in an efficient and user-friendly manner, with less concern on more specialised input. They more frequently are found on touchscreen devices, especially ones with portrait modes.
Android and iOS keyboards are what I would consider to be the gold standard, and we should probably seek to reproduce how they work.
Note that I don't include something like Hacker's Keyboard, as that contains more specialised keyboard features, such as modifier keys.
- Texting / messaging
- Simple note writing
- Dialling a phone number
- You will want to look into preediting/commiting strings using a protocol like input-method-unstable-v1.xml
- Displays on the lower half of the screen
- Ability to switch layers to access numbers & symbols
- Spell-checking (Optional)
- Mobile keyboards usually appear only when a text input is selected
- Maliit
- Squeekboard
Desktop keyboards are typically used to emulate a full keyboard, including usually having a GUI that shows a keyboard layout, complete with special keys, modifier keys and levels.
They are usually less focussed on text-specific input, and allow the user to input keybinds and other specialised input to control their computer.
I find that desktop OSKs are more useful when the input method is a mouse.
- For use on screens where the keyboard should not take up large amounts of screen estate, such as on a touchscreen TVs or projector systems
- Navigating programs using non-text input, e.g. special keys, keybinds
- Desktop Environments / Compositors
- Terminal (to send SIG events, e.g.
Ctrl+C) - Spreadsheet Program (More advanced text editing, home/end keys, etc)
- Games (using something like the arrow keys)
- Using the OSK as a GUI keyboard macro pad
- Requires a protocol that enables sending key events
- I have found that the documentation around what keymaps is a tad lacking, see the Xkbcommon docs
- Ability to float
- Ability to emulate a full hardware keyboard (including modifier keys)
- Ability to change the window's opacity
- Ability to be shown even when there isn't a text input active (i.e. an always shown setting)
- Support for differing key sizes (touchscreen users need bigger keys, whereas mice users need smaller keys so they can still see most of the screen)
- Onboard (X11, this is my personal go-to virtual keyboard)
- I have not seen any Wayland desktop OSKs
- input-method-unstable-v1.xml
- input-method-unstable-v2.xml
- text-input-unstable-v3.xml
- KDE's fake-input.xml
- External virtual-keyboard-unstable-v1.xml
- Awesome-Wayland's OSK Links
- Maliit-Keyboard
- Squeekboard
- QVK (More of a demo in it's current state, 2024/12/06)
- KDE exposes the input-method-unstable-v1.xml
(contrary to what the page claims), but it's hidden to applications by
default. You need to add
X-KDE-Wayland-VirtualKeyboard=trueinto the applications.desktopfile that is installed in the applications directory,/usr/share/applications, for KDE to accept it, and then it'll be available to select in System Settings.
Got some resources to add or tips other developers should know? Open an issue or a pull request and it can be added!
By contributing, you agree to license your contribution under the license.
- As this is an accessibility project (as OSKs are accessibility tools), any and all use-cases for an OSK should be seen as valid methods that should be accounted for and supported to some degree.
This file is licensed under the GNU FDL v1.3.