Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add first-party asset-based cursor icons to Bevy #9557

Closed
alice-i-cecile opened this issue Aug 24, 2023 · 11 comments · Fixed by #14284
Closed

Add first-party asset-based cursor icons to Bevy #9557

alice-i-cecile opened this issue Aug 24, 2023 · 11 comments · Fixed by #14284
Labels
A-Rendering Drawing game state to the screen A-UI Graphical user interfaces, styles, layouts, and widgets A-Windowing Platform-agnostic interface layer to run your app in C-Feature A new feature, making something new possible S-Ready-For-Implementation This issue is ready for an implementation PR. Go for it!
Milestone

Comments

@alice-i-cecile
Copy link
Member

FYI, winit's existing set_cursor_icon method is inadequate for games: it only supports a fixed, unstyled list and relies on OS support. In games, you often need new options (a mining pick! an axe! a sword!), and virtually always want custom assets.

This isn't something winit can fix, since it doesn't come with an asset / rendering solution. Instead, we should build our own cursor icon solution, with a "native" variant that just uses the OS ones provided by winit.

Originally posted by @alice-i-cecile in #9538 (reply in thread)

@alice-i-cecile alice-i-cecile added C-Feature A new feature, making something new possible A-Rendering Drawing game state to the screen A-UI Graphical user interfaces, styles, layouts, and widgets labels Aug 24, 2023
@alice-i-cecile
Copy link
Member Author

The first PR for this should be quite simple: just create a UI element from an image, and keep it synced with the mouse while disabling the built in cursor.

Probably have a CursorBundle or something, and then just let users manually swap the image handle as needed.

Picking-integrated reactivity can come later.

@aevyrie
Copy link
Member

aevyrie commented Nov 11, 2023

We actually need winit support to do this correctly. Relying on the rendered window to draw a cursor is never going to have the same low latency as the OS cursor.

rust-windowing/winit#3005

I've noticed other games with custom cursors have properties that prove this:

  • during game streaming, the custom cursor will become visible before the rendered output is visible
  • the cursor can leave the frame of the window when the cursor is positioned in the bottom right
  • the latency and update rate is perfect even when the game is not running amazingly well

Regardless, I agree an asset-based solution would be nice, but I wanted to point out we do need to use OS APIs to do this without jank.

@aevyrie
Copy link
Member

aevyrie commented Nov 11, 2023

More proof this is possible, navigate to this page and hover over the demo to get the rainbow icon.

https://webdesign.tutsplus.com/how-to-create-a-custom-mouse-cursor-with-css-or-javascript--cms-106724t

Scroll up so the demo is at the bottom edge of the screen. You will be able to see the custom cursor image hanging outside the bounds of the browser window. Additionally, the latency is perfect.

@alice-i-cecile
Copy link
Member Author

Definitely agree that we should hook into the OS layer.

Unsure if there's a sensible way to do this with winit itself though 🤔 It's not asset or graphics aware. Same sort of problem we ran into with window icons.

@aevyrie
Copy link
Member

aevyrie commented Nov 12, 2023

The winit API should just accept image bytes for the Custom variant. We just need a layer of indirection on the current Window cursor setting that accepts an asset handle once winit supports the feature, and a plugin that updates it with the data in the Handle<Image>.

@Friz64
Copy link
Contributor

Friz64 commented Dec 16, 2023

rust-windowing/winit#3218 has just been merged 🎉

@alice-i-cecile alice-i-cecile added the A-Windowing Platform-agnostic interface layer to run your app in label Dec 17, 2023
@djeedai
Copy link
Contributor

djeedai commented Dec 21, 2023

Came to say the same :) Is there a tracking issue for upgrading winit itself, or a process, or someone usually handling it? Note that a winit version with that feature did not release yet, but that should happen soon I guess.

@alice-i-cecile
Copy link
Member Author

#10702 just bumped the winit version to the latest release :) For the next release, whoever wants to can claim it. Just split apart the "minimal migration" from the "use new feature" for easier reviews.

@eero-lehtinen
Copy link
Contributor

Winit maintainers said that the changes won't be released in a patch version, so we might need to wait for a while. They also want to bake the feature a little bit more.

@benfrankel
Copy link
Contributor

Custom cursor tracking issue: rust-windowing/winit#3306.

@eero-lehtinen
Copy link
Contributor

eero-lehtinen commented Jun 27, 2024

Custom cursors are already released in winit 0.30 and work fine. The tracking issue is just for improvements. So this issue is ready to be implemented.

@alice-i-cecile alice-i-cecile added the S-Ready-For-Implementation This issue is ready for an implementation PR. Go for it! label Jun 27, 2024
@alice-i-cecile alice-i-cecile added this to the 0.15 milestone Jun 27, 2024
github-merge-queue bot pushed a commit that referenced this issue Aug 12, 2024
# Objective

- Add custom images as cursors
- Fixes #9557 

## Solution

- Change cursor type to accommodate both native and image cursors
- I don't really like this solution because I couldn't use
`Handle<Image>` directly. I would need to import `bevy_assets` and that
causes a circular dependency. Alternatively we could use winit's
`CustomCursor` smart pointers, but that seems hard because the event
loop is needed to create those and is not easily accessable for users.
So now I need to copy around rgba buffers which is sad.
- I use a cache because especially on the web creating cursor images is
really slow
- Sorry to #14196 for yoinking, I just wanted to make a quick solution
for myself and thought that I should probably share it too.

Update:
- Now uses `Handle<Image>`, reads rgba data in `bevy_render` and uses
resources to send the data to `bevy_winit`, where the final cursors are
created.

## Testing

- Added example which works fine at least on Linux Wayland (winit side
has been tested with all platforms).
- I haven't tested if the url cursor works.

## Migration Guide

- `CursorIcon` is no longer a field in `Window`, but a separate
component can be inserted to a window entity. It has been changed to an
enum that can hold custom images in addition to system icons.
- `Cursor` is renamed to `CursorOptions` and `cursor` field of `Window`
is renamed to `cursor_options`
- `CursorIcon` is renamed to `SystemCursorIcon`

---------

Co-authored-by: Alice Cecile <alice.i.cecile@gmail.com>
Co-authored-by: Jan Hohenheim <jan@hohenheim.ch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Rendering Drawing game state to the screen A-UI Graphical user interfaces, styles, layouts, and widgets A-Windowing Platform-agnostic interface layer to run your app in C-Feature A new feature, making something new possible S-Ready-For-Implementation This issue is ready for an implementation PR. Go for it!
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants