Add zooming & translation controls to Axis3

[ffreyer]

Description

Now that #3958 has been merged we can implement zooming and translation for Axis3 without data spilling out of he Axis limits.

Old Description

I don't aim to work on this immediately. Instead I'm hoping that this pr can spark some discussions on how zooming and translations should work with Axis3, so that the end result is more polished.

Some questions:

• What point should zooming relate to?
  • The center?
  • A point offset from the center based on the mouse position?
  • A data point under the mouse?
• Should up/down mouse movements strictly map to z-axis translations, and left/right mouse movements strictly map to xy-axis translations? Or should they freely map to the corresponding (x, y, z) vectors?
• How should translation speed scale with zoom? Should we set a maximum based on data limits to avoid losing sight of data?
• What things should we port from Camera3D/LScene? E.g.:
  • Directional restrictions for translations when pressing x/y/z and translating
  • Focus on clicked object/point on object (left alt + left click)
  • Recenter hotkey (left ctrl + left click)
  • Keyboard controls

I extended the following interactions to work with Axis3:

• ScrollZoom which comes with x/y/zzoomlock and x/y/zzoomkey Axis3 attributes like Axis does, and additionally a zoommode which controls whether zoom is centered on the Axis3 area (default) or on the cursor (1)
• DragPan which comes with x/y/ztranslationlock and x/y/ztranslationkey rather than ...pan... as pan technically refers to a rotation of the camera.
• LimitReset

(1) "zoom centered on the cursor" really means that we want to keep a point under the cursor at the same location as we zoom in or out. With perspective projection in 3D this depends on the depth value of the point, which changes as we zoom. Without knowing exactly where that point is, we cannot fulfill that condition. And even if we ray-cast to find the coordinate, another point at a different depth that initially overlapped will move away as we zoom. Because of that I think it's better to just zoom towards the well-defined center.

TODO:

• update documentation
• maybe add interaction for "center on clicked object"
• Reuse naming scheme and interactions from Axis
• Add tests, check failures
• fix axis reversal
• fix CairoMakie text clipping
• Discuss whether clipping related changes count as breaking or bug fixes I think it doesn't actually change anything...?
• test every possible viewmode with different perperspectiveness, axis reversals and aspect settings

Related issues: #3792, #1655, #1667

Type of change

• New feature (non-breaking change which adds functionality)

Checklist

• Added an entry in CHANGELOG.md (for new features and breaking changes)
• Added or changed relevant sections in the documentation
• Added unit tests for new algorithms, conversion methods, etc.
• Added reference image tests for new plotting functions, recipes, visual options, etc.

[MakieBot]

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(fig)

	using	create	display	create	display
GLMakie	4.47s (4.42, 4.53) 0.04+-	108.30ms (105.79, 112.15) 2.56+-	469.34ms (450.90, 490.37) 14.54+-	9.28ms (9.02, 9.58) 0.23+-	26.12ms (25.97, 26.31) 0.12+-
master	4.50s (4.46, 4.59) 0.04+-	108.03ms (104.61, 112.49) 3.05+-	475.31ms (468.93, 487.40) 6.21+-	9.52ms (9.18, 9.74) 0.18+-	26.28ms (25.98, 26.85) 0.29+-
evaluation	1.01x invariant, -0.03s (-0.72d, 0.21p, 0.04std)	1.00x invariant, 0.27ms (0.09d, 0.86p, 2.80std)	1.01x invariant, -5.96ms (-0.53d, 0.35p, 10.37std)	1.03x faster ✓, -0.24ms (-1.18d, 0.05p, 0.21std)	1.01x invariant, -0.16ms (-0.72d, 0.21p, 0.21std)
CairoMakie	4.23s (4.19, 4.28) 0.04+-	106.76ms (105.90, 108.51) 0.87+-	168.54ms (166.98, 171.73) 1.68+-	9.68ms (9.35, 9.79) 0.15+-	1.15ms (1.13, 1.17) 0.02+-
master	4.23s (4.17, 4.28) 0.04+-	107.60ms (105.59, 113.14) 2.52+-	167.65ms (166.43, 168.93) 0.93+-	9.72ms (9.35, 9.95) 0.19+-	1.16ms (1.14, 1.18) 0.02+-
evaluation	1.00x invariant, -0.0s (-0.04d, 0.95p, 0.04std)	1.01x invariant, -0.84ms (-0.45d, 0.43p, 1.69std)	0.99x invariant, 0.89ms (0.66d, 0.25p, 1.31std)	1.00x invariant, -0.04ms (-0.22d, 0.68p, 0.17std)	1.00x invariant, -0.0ms (-0.27d, 0.62p, 0.02std)
WGLMakie	4.68s (4.66, 4.70) 0.02+-	106.33ms (105.33, 108.51) 1.08+-	8.97s (8.89, 9.08) 0.07+-	11.36ms (11.17, 11.50) 0.15+-	117.89ms (113.84, 119.79) 2.04+-
master	4.71s (4.66, 4.83) 0.06+-	106.75ms (105.45, 109.86) 1.47+-	9.31s (9.22, 9.41) 0.08+-	11.66ms (11.30, 13.07) 0.63+-	117.50ms (114.12, 124.49) 3.25+-
evaluation	1.01x invariant, -0.03s (-0.71d, 0.23p, 0.04std)	1.00x invariant, -0.43ms (-0.33d, 0.55p, 1.28std)	1.04x faster ✓, -0.35s (-4.73d, 0.00p, 0.07std)	1.03x invariant, -0.3ms (-0.65d, 0.26p, 0.39std)	1.00x invariant, 0.39ms (0.14d, 0.79p, 2.65std)

[asinghvi17]

Zooming for Axis3 is basically just shrinking finallimits, right?

[ffreyer]

I don't know yet. In the prototype code it is, but I don't know if that's a good way to do it. that code was really just to test if clip_planes allow me to cut off plots outside the Axis bbox while keeping the Axis decorations.

[ffreyer]

Not sure in what context this was crashing but it's not crashing anymore on my laptop or PC.
Without this turning clip_planes on once will turn them on forever, with random data. This causes plots to disappear randomly. Maybe this should be a hotfix instead of being part of this pr...

[ffreyer]

This is done from my side. Tests fail due to scatter markers not getting cut off by scene boundaries anymore and some negligible sub-pixel movements of axis decorations.

Something that may still be worth discussing is how Axis3 content and decorations interact with the region layouting assigns to the Axis3. Currently the content is locked to that region, i.e. if you do mesh!(ax, ...) that mesh is clipped at the boundary assigned by the layout. Decorations are not constrained to this region, because they weren't before and I tried to make this less breaking. (This is why sometimes a x/y/z labels will end up drawing over another layout element)

The new refimg marks the area the content is restrained to (i.e. computedbbox) with pale orange. So here the cat would be cut off at the red border. (The blue region is with protrusions subtracted and the different cells are changes in aspect and xreversed)

[ffreyer]

Something that may still be worth discussing is how Axis3 content and decorations interact with the region layouting assigns to the Axis3. Currently the content is locked to that region, i.e. if you do mesh!(ax, ...) that mesh is clipped at the boundary assigned by the layout. Decorations are not constrained to this region, because they weren't before and I tried to make this less breaking. (This is why sometimes a x/y/z labels will end up drawing over another layout element)

Example for this:

using FileIO, GLMakie
cat = load(Makie.assetpath("cat.obj"))

fig = Figure()
Box(fig[1, 1:3], color = :lightblue)
Box(fig[3, 1:3], color = :lightblue)
Box(fig[2, 1], color = :lightblue)
Box(fig[2, 3], color = :lightblue)
ax = Axis3(fig[2, 2], viewmode = :free)
mesh!(ax, cat, color = :orange)
fig # and some zooming/interacting

[SimonDanisch]

Looks good to me! Not sure if @jkrumbiegel has time to look at this, but may make sense?

[jkrumbiegel]

Looks pretty good @ffreyer! The only thing I didn't like while trying it out was that if you're zoomed and panned to some position off-center, you lose your position quickly when further scaling or rotating because the reference point is still the center of the axis. I think it would be better to see if any plot objects in the axis can be picked, and if so, use the picked point as the pivot.

[ffreyer]

I think I avoided doing these kinds of things because a lot of the times I tried to be smart and fancy with Camera3D it backfired.

Some options for rotating are: (for viewmode = :free)

• rotate around axis center (current) - this is fairly easy to do, you just need to shift your coordinate system to the right origin
• rotate around scene center - this is also easy, and would cause the axis to rotate like in LScene. I.e. if you translate it away from the center, it rotates on a sphere surface around the center.
• rotate around cursor is tricky because of perspective projection...
  • rotate around ray-cast intersection between cursor and content - both the content and cursor move so what's under your cursor will change.
  • rotate around settable point - Camera3D can do this with left_alt + left_click. I think it's a decent solution but I'm not sure if it's good for general use. If you're looking at surface details of a mesh, rotating around a point on the mesh sounds useful. When you have multiple meshes or you care about the mesh as 3D object, you would probably prefer rotating around the center of the mesh.

Zooming is somewhat similar. You can:

• zoom relative to the axis center (current) - easy to do, same as rotation, but not what you want
• zoom towards the scene center - also easy like the respective rotation. Camera3D can do this with cam3d!(a, zoom_shift_lookat = false). Also not what you want
• zoom toward cursor is tricky...
  • zoom towards cursor based on offset from center - is what zoom_shift_lookat = true does. I find this annoying when zooming out. I often end up pushing stuff I want to look at off-screen or into a small space when zooming out. Also still allows zooming past the object
  • zoom towards point under cursor - doesn't have issues with cursor/content movement and would allow for zooming to never enter/pass the content. But now zoom speed can fluctuate depending on what your cursor hovers. Also has the zoom-out problem

I would be fine adding a switch for scene vs axis centered rotation and zooming, and also separating the axis center from the rotation + zoom center and adding extra interactions to control it (like Camera3D's alt-click, a secondary translation, a secondary reset). Those sound like relatively "dumb" interactions to me and therefore things you can relatively easily reason with.

[ffreyer]

I realized that I could also just reuse the viewmode != :free translation to allow you to center the plot. Now it's setup like this (with viewmode = :free):

• right drag translates the content of the axis, allowing you to move the thing you want to rotate around and zoom towards into the center
• ctrl + right draw translates the entire axis (in the window)
• shift + left click now exists independently of ctrl + left click and re-centers content in the scene. ctrl - click reverts zooming, axis translations and rotations (maybe these tasks should be distributed differently?)
• alt + click has been added to center what the cursor hovers

Also did a bit of cleanup and fixed a rendering bug caused by near being too small:

I also noticed that #3584 is reappearing here, because frame lines are projected manually

[ffreyer]

I also noticed that #3584 is reappearing here, because frame lines are projected manually

I took the easy way out and just moved them to 3D space/scene. Technically this should also be an issue for ticks, tick labels and axis labels, but I think it's much more rare.

[ffreyer]

Tick alignment with frame lines got worse when I moved the frame to data space. So I spent too long trying to figure out why.

I thought it might be a Float32 vs Float64 thing, so cleaned up the remaining Float32 types in Axis3. Didn't help. I tried forcing lines to convert on the CPU to make sure conversions don't happen till the end. Didn't help.

Then I got frustrated and moved the CairoMakie line point projection to Makie so I could revert moving the frame to data space without having clipping issues. And the alignment issues were still there. Then I tried using that code for ticks as well and it finally went away.

So i tried to figure out why and started comparing. Didn't notice the resolution - 1. Noticed that clip space points were very different, due to the CairoMakie code clipping to a -1..1 box. So I changed things up to avoid it and there was still a 0.5-1px difference. And then I finally noticed this line, changed it and I the issue seems to be fixed now.

Anyway, fixes #3302.