Implement kms backend?

[goddessfreya]

Right, so there was some chatter about implementing a kmsmod backend like SDL's. Mouse input could be provided by a library outside of winit that uses evdev.

Such a backend would allow us to run applications in the tty, without the presence of X11/Wayland/DirectFB, which I'm sure would open up a lot of silly possibilities. To use alacritty as an example, such a backend would allow it to replace similar projects like kmscon, which I think would kind of be nifty.

@chrisduerr What's your opinion on running alacritty in the tty?

[Osspial]

Could you provide a link to the SDL discussion and what exactly kmsmod is? I can't seem to find either from some quick google searches.

[chrisduerr]

@chrisduerr What's your opinion on running alacritty in the tty?

Running Alacritty itself in the TTY seems a bit pointless tbh, since it builds on GUI and GPU Acceleration at its foundation, but doesn't provide much functionality when it comes to window management (it instead outsources that to the WM).

However, it would be possible to rewrite the renderer of Alacritty for a TTY backend, but at that point it would probably be easier to just skip glutin/winit completely.

However I've never had any interest in replacing my console, so maybe other people have different opinions here. I don't think it wouldn't be fun to play around with, but I would never upstream support for a PR that implements this since it's just a load of mostly unnecessary (to me) complexity.

And I feel like that might be similar for winit/glutin in general. I'd be curious why this was requested. Which kind of application should this support and who's the target audience? Since many features of winit/glutin probably don't exist there anyways, would it maybe be better to just write a completely separate toolkit? What can winit/glutin provide here that wouldn't have to be written from scratch?

Also additional platforms of course always mean additional work and maintenance overhead. Testing in CI might be difficult and it is probably even more work than just testing linux with a display server. So who would be willing to maintain it? Initial implementation is one thing, but often maintainership is even harder than the initial support.

Of course these are just my two cents, making guesses about a chat conversation I haven't been part of, but in general I'm not immediately in love with the idea.

[goddessfreya]

Could you provide a link to the SDL discussion and what exactly kmsmod is? I can't seem to find either from some quick google searches.

Discussion took place on discord. Go to the gamedev channel on the rust community discord and search for there's one thing winit doesn't do as far as i know that i would like to: run straight out of a terminal.

https://wiki.archlinux.org/index.php/Kernel_mode_setting

[goddessfreya]

but doesn't provide much functionality when it comes to window management (it instead outsources that to the WM).

You're in luck, nor does kmscon.

And I feel like that might be similar for winit/glutin in general. I'd be curious why this was requested. Which kind of application should this support and who's the target audience?

This was suggested for the shits and giggles.

Since many features of winit/glutin probably don't exist there anyways, would it maybe be better to just write a completely separate toolkit? What can winit/glutin provide here that wouldn't have to be written from scratch?

Well the idea was to get existing programs using glutin working in the tty with minimal changes to them. It could be a separate library, of course, but it's going to end up exposing the same api as winit/glutin. Also, most of the features should be present, if we use kmsmod with evdev. If the computer supports hardware cursors, mouse support would even be present. If not, a software impl could probably be schemed up in a separate library.

Testing in CI might be difficult and it is probably even more work than just testing linux with a display server. So who would be willing to maintain it?

Currently we don't test programs in the CI, just make sure they compile. I'd argue testing this backend would be easier than others. Ctrl-alt-F2, cd into glutin, then cargo run --example window, or something like that.

Also additional platforms of course always mean additional work and maintenance overhead. [...] Initial implementation is one thing, but often maintainership is even harder than the initial support.

Agreed. The person who submits it should proably also maintain it. (Altho I suspect this backend will require minimal maintenance.

[elinorbgr]

For what it's worth, this is very similar in scope to what smithay does, if you only keep the backend part and disable the wayland-clients-handling features.

[NickeZ]

If I understand correctly, this would be very cool for security sensitive "embedded" and/or "kiosk" applications where you don't want to run the X11 stack or other c-software full of possible leaks.

[Lokathor]

Yeah the point is that low end devices (raspberry pi, odroid, etc) can have things to display but not want to put an x11 session in RAM.

[aleksander]

So something like kmscube would be possible? Cool!

[cheako]

kmscon isn't maintained anymore, I'll just call this virtual console? To write an abstraction layer for an application to choose from winit and an internal handler for virtual console might be just the same as writing a eventloop for winit. I'd like to get a feeling for how well something like winit/src/platform_impl/linux/vc would be received?

[Lokathor]

SDL2 supports DirectFB in their docs and I was able to get it to open a fullscreen window on my rpi without turning on X. Was it probably going through DirectFB when it did that? Should we support DirectFB as a driver?

[goddessfreya]

kmscon isn't maintained anymore,

kmscon != kms.

[inodentry]

Everyone is going on about using this for alacritty for a kms-based console. While that is cool, it is just one use case. I'd like to point out other awesome use cases for this functionality:

1. GUIs for embedded systems. If you only have one single fullscreen application that should have total control of everything and the user should directly interact with. It would be nice for that application to run without requiring X11 or a wayland compositor or anything of that sort. Just launch it and it takes care of its display and rendering and input on its own however it likes (using this proposed winit functionality).

2. Games. This would allow to develop a game that effectively has direct access to the display, GPU, input, etc. This would remove any overhead that would otherwise come from the windowing system protocol / compositor and result in the best possible performance and latency.

[nezticle]

I see this as very analogous to what we have in Qt with the EGLFS platform: https://code.qt.io/cgit/qt/qtbase.git/tree/src/plugins/platforms/eglfs
This itself has a lot of complication because there are a couple of different approaches to actually get the EGLDisplay and EGLSurface (fbdev, drm/kms), and then further there are sometimes vendor specific tricks you have to do as well. In the case of the EGLFS plugin there are additional plugins to handle this, which is much more complicated than the current platforms support. So maybe check out EGLFS for some inspiration for how this could look/work here.

[NathanRoyer]

Hi, I'm here following my duplicate PR: #1865.

there are a couple of different approaches to actually get the EGLDisplay and EGLSurface (fbdev, drm/kms)

fbdev and its doc are outdated now; DRM/KMS is the currently advised way to go, because while fbdev merely allowed to manipulate a framebuffer, DRM intends to provide in-kernel support for more GPU control.

If I understand correctly, this would be very cool for security sensitive "embedded" and/or "kiosk" applications where you don't want to run the X11 stack or other c-software full of possible leaks.

That's exactly my case, I want to use programs like Servo as a kiosk app in an embedded environment which lacks a compositor. Servo and it's major component webrender rely on winit.

As stated in #1865, I have working rust code demonstrating the use of DRM/KMS, so I'm going to try to implement this backend.

[cheako]

There is also the use case of writing compositors.

[NathanRoyer]

I have some sort of draft and it compiles, but in the end I think I will have no use for it with servo. My implementation replaces a lot of x11/wayland code instead of sitting beside it (platform_impl/linux is such a mess) so I consider it bad and I doubt I will make it clean in the future. However my test raised some questions about how this should be done; Should there be:

• one window extending over all monitors (what about the geometry ?)
• one window per monitor (easy, probably the way to go)
• a virtual space where windows lay and monitors peak at a rectangle inside this virtual space (what I implemented) → this impacts performance as monitors usually don't have the same refresh rate

Also raw_window_handle has to pass some DrmHandle I added in my implementation, representing an EGL surface. DrmHandle has two fields (card + resource handle) and I find it quite dirty. winit clients draw on the surface using egl. This is one way to do but there must be others.

[cheako]

I don't think there should be a 1:1 mapping to the concept of Windows, likely another abstraction Displays or Cards that implements a common trait with Windows would be the way to go. If an application is targeting virtual consoles it's likely they won't be interested %90 of any Window abstraction. In the interests of having small purpose dependent crates I believe that higher level abstraction(*s) would be used for applications that want to target both.

• There would be cases were the decision is made at compile time and other applications would want to make that decision at runtime, so there is room for two crates that provide a single abstraction.

[StratusFearMe21]

btw, here is a RAM usage breakdown for my install of Arch Linux using this fork of Alacritty

• In TTY: 290MB
• In DWL and Alacritty: 466MB
• In Alacritty KMS/DRM mode: 308MB

[chrisduerr]

The mouse cursor is invisible

Considering Wayland already draws its own mouse cursor iirc, it should be relatively trivial to do the same, right?

[StratusFearMe21]

True, however, Wayland can change how the cursor is drawn depending on the graphics library in use. Winit cannot since it's graphics library agnostic.

[kchibisov]

True, however, Wayland can change how the cursor is drawn depending on the graphics library in use. Winit cannot since it's graphics library agnostic.

Cursor rendering on Wayland is client side. And most clients use some libs to load cursor and draw it.

You can use https://github.com/esposm03/xcursor-rs to achieve that, since that's what wayland-rs is using.

[cheako]

Wayland/X/kms was a compile time choice, so I don't see the downside of being forced to deal with the *inconvenience of not having a Window abstraction. The code changes would either be minimal or substantial, in the latter case I doubt a stub abstraction layer would be sufficient.

• The type system enforcing the differance in environments sounds rust like to me.

[vanfanel]

@StratusFearMe21 Having Alacrity on KMS/DRM is a dream come true for me. Thanks, really, for the work you put into this!

Two questions come to my mind:
-Is it really impossible to have other KMS/DRM apps to be launched from that terminal? Apart from the obvious fact that alacrytty/kmsdrm is owning the KMS/DRM system: can't it "let it go" while the KMS/DRM program runs?
-Is it possible to use shaders with this? Like in, just imagine a TTY console with proper scanlines + shadow mask + curvature...

[StratusFearMe21]

1. Im not entirely sure on that. Ill have to do some more research into that.
2. If you're willing to mess about with the alacritty renderer, yes, shaders are absolutely possible

[vanfanel]

@StratusFearMe21 Please, PLEASE keep me/us all informed about your findings with regards to point 1... just imagine having an advanced, accelerated TTY console on GNU/Linux at long last. It's long overdue so I think that what you are doing is very important.

By messing with alacritty renderer, do you mean adding loadable shaders support?

[StratusFearMe21]

Yes. The best part about adding that is that it would apply to all platforms.

[mraerino]

i updated @StratusFearMe21's PR to match current master: #2795

[notgull]

This issue came up in our weekly winit maintainer's meeting. Our question is: how badly do people want this?

If one of our stakeholders (egui, bevy) really wants this, I'm happy to write and maintain a DRM/KMS backend. If someone is willing to step up and become a maintainer, we can have them write and maintain the backend. But only if they are committed to maintaining this backend.

Otherwise, it's a lot of additional work for a platform that I doubt will be popularly used. Thus far most of the rationale is "it'd be cool to use Alacritty as a TTY". But it would add a lot of additional maintenance just for that.

[cheako]

You won't get users until you have an implementation... This chicken-egg style thinking is self-destructive. Unless you are advocating that KMS is deprecated and shouldn't have any users, then the use cases for winit are the use cases for KMS.

[kchibisov]

Unless you are advocating that KMS is deprecated and shouldn't have any users, then the use cases for winit are the use cases for KMS.

KMS is basically writing wayland compositor, my opinion that if you're into writing wayland compositor, better use smithay or wlroots and run winit as wayland app inside. If we're talking about traditional kms stuff, it's also not really portable to anything other than linux and freebsd, the rest is far more challengeable to get right.

Updating it also requires special knowledge for little to no benefit, since the primary consumer for simple kms stuff will be kiosk application, but you can use cage for that just fine.

I'd welcome such backend out of tree to exist after winit 0.31.0.

[cheako]

video players, FPS/games where latency is king, login managers, boss screens and screen savers... I'm sure I could think of more.

[kchibisov]

Modern wayland compositors have low latency and soon can do async page flips. They also direct scanout arbitrary surfaces and not just fullscreen. To write login manager you can use smithay, I did that myself, took less than 1k LoC and I hand rolled drm swapchain.

Screen savers for kms?

People who interested can write such backend out of tree after 0.31.0 as I said. Having special purpose backends in tree is not what I really want to see, since I'm always the one dealing with all of that in the end.

[daxpedda]

Having special purpose backends in tree is not what I really want to see, since I'm always the one dealing with all of that in the end.

I'm not sure what exactly this means, but if we have a reliable maintainer, in this case @notgull, I don't see a reason why not to have this in-tree. If you somehow "end up maintaining it anyway", you can kick it out then.

[chrisduerr]

If you somehow "end up maintaining it anyway", you can kick it out then.

If only making software was that simple.

[notgull]

To clarify, I only offered to maintain this theoretical DRM/KMS backend if one of winit's major users wanted it. As far as I know, this isn't the case.

The reason being is that it's actually quite a bit of work to maintain a winit backend. Especially for a backend that would involve a substantial amount of new code, you also have to keep up with new designs and bug fixes.

Not to mention, it ends up being a lot of work for other maintainers trying to implement new design patterns. I'm not a maintainer for any backend except X11, but I have to know all of the other backends moderately well in order to implement new features or do any refactoring.

Anecdotally, we've already had two people step up to be maintainers and then bail out for what I assume to be the realization that maintaining this backend is more than they bargained for. So it's definitely work that can't be done enthusiastically.

If someone else wants to step up be my guest, but we'd expect you to maintain it for at least a while.

[cheako]

Sorry I didn't mean for this to turn into an argument, I just wanted to clarify that the one argument about not having users sounds fallacious and might not be a good argument. It's obvious and unimpressive that nobody is using code that hasn't been written and using the lack of ppl trying as a justification.

[kchibisov]

I'm only against having in tree because it's the hardest backend to get right and maintain as well as bring to some feature parity. But being it out of tree won't make any difference to it, other than not being automatically updated by core maintainers.

[ids1024]

There are a couple issues here.

• It seems like as things are structured now, a new backend adds a bit much maintenance burden, even if it's a simple backend. Improvement to that will make things like a KMS backend easier to support, in tree or in a separate crate.
• Some aspects of KMS are different from a normal windowing backend, and don't map too simply to Winit.
  • How should multiple monitors be handled? Potentially connected to multiple graphics cards, which has implications if you want to use accelerated graphics.
  • How is mode-setting done? Does it just use the default resolution of the monitor, or does the backend need to expose some way to configure this?

I wonder if it might be good to create a Rust library for handling framebuffers, which could provided a somewhat higher level API over drm/kms, and other backends if anyone cares much (Linux's older fbdev API, wscons for some hardware on NetBSD/OpenBSD, Redox framebuffer). That library could provide raw-window-handle support, and perhaps a winit-compatability layer could be built on top of that, along with something for handling input.

[cheako]

Winit would be useful to run KMS apps nested. It's not really material if winit's API is structured to allow virtual console as a backed or if winit is a backend for a higher level API, but that is the heart of this issue. It's confusing to think that winit would become a low level backend alongside KMS, would one assume that a KMS backend would be raised to unorthodox heights as a result of this arrangement?

[kchibisov]

winit is moving towards winit-core and backends in external crates, thus you'll be able to write your backend for whatever you want (probably even tui).

[daxpedda]

We have agreed that there is currently no sufficient external (and internal) motivation to maintain this in-tree.

So until this changes, the goal is to support it out-of-tree, which is covered by #3433.

Thanks everybody for your input so far!

[madsmtm]

I'd like to add: While I recognize that this could be useful in other contexts, it really does sound like most people want this in Alacritty. So in the meantime, until we can do out-of-tree backends, you could also consider supporting this directly in Alacritty, i.e. without doing a Winit fork.

[kchibisov]

alacritty is not really interested unless it's in winit and in general, not sure that we that much interested in putting extra work here, unless winit will do backend split and we'll be able to remove the amount of cfg's we have flying around.

[cheako]

Am I missing the logical argument being made? I was thinking perhaps it was that alacritty is not a true Scotsman, but that's not at all what's being said. Why should all the other projects care what the operators of alacritty want? That we could all agree a terminal would be a good use case for the proposed technology is the important part. How ppl feel about the prospect in individual cases(Scotsman or not) should not be considered, because there are more than a couple and knocking them down one by one would be the no true Scotsman fallacy.