Mouse location within viewport rendered to image

[ThomasAlban]

I am writing an editor using bevy, which uses egui dock and renders the camera to an image which is displayed in a tab. How do I correct the mouse position of bevy_mod_picking so that it is correct? I need some way of giving bevy_mod_picking mouse coordinates from egui. Thanks

[aevyrie]

You will need to (a) write a shim, or (b) write your own picking backend: https://docs.rs/bevy_mod_picking/latest/bevy_mod_picking/backend/index.html

To support picking over an image, you will need to create a shim that takes the pointer's location on the image from egui, and sends it to the picking plugin.

• in egui, on your image, get the pointer's position
• override the pointer's PointerLocation - instead of the window render target, set it to your image's rendertarget

This should be enough to inform the existing backends of where your pointer is. Because they know the pointer is over an image instead of a window, they can find the camera associated with that image and use it for raycasting, for example. The downside of this approach is that it is hijacking the pointer from the window, and putting it on an image, which means that pointer cannot pick things in the window while it is over the image. This sounds like a non-issue for your use case.