Touchscreen gestures

[minego]

I've recently purchased a minisforum v3 tablet, and I'm very happy with how niri runs on it, but it is rather unusable in tablet mode, since I have no way to switch windows or workspaces.

I'd love to be able to bind support for gestures on the touchscreen, in a similar way to the trackpad. As a starting point I think the same gestures that the trackpad supports would be sufficient, but being able to do others like swiping from a screen edge, would be very useful as well.

I hope to dig into the source and try to see if I can make this work, but I am not sure where to start.

[YaLTeR]

Yeah, it would be good to have some touchpad gestures (and also finish missing touch input in the screenshot UI). However, I think this depends on gesture bindings, as I've got feedback to not add 3-finger touch swipes (at least, by default) because that prevents apps from using 3 finger touches themselves, which can be limiting (think a game console emulator where you need to press 3 buttons at once). For the gesture bindings, I don't have a design finalized, only some thoughts here: #372

[minego]

Personally I feel like applications wanting to use it is rare enough that I'd prefer to have it on by default... But maybe that is a more common use case than I thought? It feels to me like using gestures to navigate is more important than other use cases it may conflict with though. I mean, if I have to use a keyboard and mouse to start a game anyway... Or, perhaps it would make sense to enable them by default, but disable them for full screen windows or based on app rules? I don't know. Mostly just brain dumping now.

…

On Sat, Jun 22, 2024, at 1:38 PM, Ivan Molodetskikh wrote: Yeah, it would be good to have some touchpad gestures (and also finish missing touch input in the screenshot UI). However, I think this depends on gesture bindings, as I've got feedback to *not* add 3-finger touch swipes (at least, by default) because that prevents apps from using 3 finger touches themselves, which can be limiting (think a game console emulator where you need to press 3 buttons at once). For the gesture bindings, I don't have a design finalized, only some thoughts here: #372 <#372> — Reply to this email directly, view it on GitHub <#463 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADSRCFVODEGGI4SNZ3NNLTZIXHDRAVCNFSM6AAAAABJXTNGQOVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TQNBYGY2DC>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[minego]

I just had a thought... For my purposes, if I could tap a button on my bar to switch to a mode where swiping with 1 finger acts like the gestures, I'd be pretty happy... Not perfect, but a good start.

I could add a button in waybar to send a niri msg command to turn on the mode, then switch windows or workspaces, etc... Right now I have no way at all to do these things without the keyboard attached, since virtual keyboards can't trigger niri bindings.

I'm hoping to find some time to play with the niri source and see if I can experiment with this, but that will take a bit of learning (I don't know rust well...) and I don't know when I'll find the time.

[minego]

I spent a while looking at how the 3 finger touchpad gestures work, to try to see if I could make them work with a touchscreen... I was hoping I could add an action that can be triggered with niri msg action to turn on or off a mode where any pointer event would be treated like a 3 button gesture.

But, it doesn't seem to be as generic as that... I'm assuming that is because of how Smithay or libinput works.

I may make another attempt later...

[YaLTeR]

FWIW the same 3 finger gesture is currently hooked up to both the pointer input (Mod+MMB) and to the touchpad input. (With a slight difference to how the gesture works to account for the different input device.) For a touchscreen the deceleration multiplier will be slightly different, otherwise it'll be the same as touchpad.

This is all completely orthogonal to adding an action like this of course. But, I'm wondering if it should even be an action in the first place? Rather than just a gesture bind (which needs finishing the design for those).

[minego]

FWIW the same 3 finger gesture is currently hooked up to both the pointer input (Mod+MMB) and to the touchpad input. (With a slight difference to how the gesture works to account for the different input device.) For a touchscreen the deceleration multiplier will be slightly different, otherwise it'll be the same as touchpad.

This is all completely orthogonal to adding an action like this of course. But, I'm wondering if it should even be an action in the first place? Rather than just a gesture bind (which needs finishing the design for those).

Oh, really? I'll have to dig more then!

Yeah, an action may not be the right way to do this but, for a first step for experimenting it will be enough. Right now my plan is to have a button on my bar that can trigger the action, but long term some gesture would be much better. Perhaps something like pinch in with 4 or 5 fingers...

[itshog]

I'd really like to be able to use the same three fingers scrolling gestures currently implemented for touchpad input when using my convertible laptop in tablet mode: the scrolling paradigm Niri uses is IMO already a better option than "classic" tiling for tablets and convertibles, and having the option to scroll or change workspace with a three fingers swipe could be a massive improvement. We could finally have an intuitive, elegant and user friendly interface for convertible laptops on linux, without relying on dozens of buttons in the bar of the wm just for basic functionality.

I know about lisgd (the program used for touch gestures on Sxmo if I'm not mistaken), and probably with some tweaking it could be made to work, but I believe that a native solution which mimics the already amazing touchpad gestures has some value. Also, I couldn't find in the output of niri msg action a command that works the same as the three finger horizontal swipe to use with lisgd (but maybe I just didn't notice it).

I apologize beforehand in case there are technical reasons preventing the implementation of touchscreen gestures, like missing functionality in Smithay or something else.

[itshog]

Another option would be requiring gestures to start from a screen edge like in Android with the "back", "show all open apps" and "switch focused app" actions, but this doesn't completely solve the problem. For the record, the problem is present in Android too: for instance, if you have the web browser urlbar on the bottom of the screen, a right/left swipe on the urlbar should switch tab, but in lots of cases it ends up getting registered as a gesture and it switches application instead.

[YaLTeR]

Yeah, also apps should be able to use left and right edge swipes too I feel. But, I think bottom and top edge swipes can be reserved for the compositor (similarly to how they are on tablet/phone OSes). In GNOME, a top edge swipe is already reserved to unfullscreening, and bottom edge swipe to bringing up the on-screen keyboard if I'm not mistaken?

[itshog]

Maybe this could be left up to the user, by having separate touch-swipe-3 and touch-swipe-edge-3 options. With the first, the user knows that this may conflict with applications, while the second is a little bit less ergonomic on big screens but resembles more the way mobile operating systems usually work (and in this case it could be possible to have also touch-swipe-edge-1 and touch-swipe-edge-2).

[itshog]

So this way we could combine both approach: gestures from anywhere on the screen starting with a minimum of 3 fingers, and edge gestures starting with a minimum of one.

[YaLTeR]

Will need to see yeah.