Globally available world coordinates relative to pointer position

[pedroalmeida415]

The current project I'm working on has multiple components that require the world coordinates represented by the intersection of a ray on a plane that is at origin and is normal to the camera (which is stationary), with current pointer position as direction.

However, none of these components has a full-screen PlaneGeometry for me to state.raycaster.intersectObject() to, thus I needed a way in which I could get said world coordinates without relying on any object.

This is possible by using a Plane along with state.raycaster.ray.intersectPlane, but it still bothered me that I had to do the same setup multiple times on each component.

The solution I came up with uses an Object3D and its position as a proxy to provide the coordinates to any component just by retrieving it from the scene:

// src/components/canvas.tsx
import { Canvas as CanvasImpl, useFrame} from '@react-three/fiber'

export const Canvas = () => {
  return (
    <CanvasImpl>
      <Pointer3D />
      {/* 
         other components...
      */}
    </CanvasImpl>
  )
}

const Pointer3D = () => {
  const pointer3DRef = useRef<Object3D | null>(null)

  const normalPlane = useMemo(() => new Plane(new Vector3(0, 0, 1), 0), [])

  useFrame((state) => {
    if (!pointer3DRef.current) return
    state.raycaster.ray.intersectPlane(normalPlane, pointer3DRef.current.position)
  // negative index so that this runs before any other loop
  }, -1)

  return <object3D visible={false} matrixAutoUpdate={false} frustumCulled={false} ref={pointer3DRef} name='Pointer3D' />
}

// src/components/some-component.tsx
export const Component = () => {
  useFrame((state) => {
    // retrieve and use the world coordinates (update uniforms, etc...)
    const pointer3D = (state.scene.getObjectByName('Pointer3D') as Object3D).position

   //...
  })

return //...
}

This implementation can be further improved to support camera movement and rotation with OrbitControls by dynamically updating the plane relative to the camera, here is an example codesandbox by @mattrossman

References

• Meaning and usage of viewport, size, aspect, and mouse #857
• https://stackoverflow.com/questions/13055214/mouse-canvas-x-y-to-three-js-world-x-y-z
• https://stackoverflow.com/questions/11036106/three-js-projector-and-ray-objects/23492823