UI button interaction for vision os

Scene setup required

XR origin, ray cast, plane management

In scene Start, the GameObject MixedRealityPlayspace needs to have an XROrigin, the ARRaycastManager (from AR Foundation), ARPlaneManager (from AR Foundation), and InputActionManager (from XR Interaction). The InputActionManager should only have the XRI PolySpatial Input Actions (from XR Interaction Samples from the VisionOS sample). The input action is included in the project.

Camera setup

Two Tracked Pose Driver game objects should be included on the main camera: the first one from the Input System package, the second one is system-level (coming from XR legacy input helpers package, probably not needed). They both set up XR device position/rotation/state information for the Input System.

XR rig

The XRI_simple_rig prefab from the XR Interaction samples needs to be added to the Root game object, removing the Scene Camera. Since XR origin and AR session are already initiated somewhere else, they can be removed / deactivated. This leaves only the XRInteractionManager component here - and the child game objects setting up input mappings.

Primary Interaction Group and Secondary Interaction Group Variant can have the XRInteractionManager directly assigned:

Poke and NearFarInteractors should allow for UI interaction:

Set up the PolySpatialInputAction mappings in project settings

PolySpatial and XRInteraction samples both provide InputMaps (in Assets/Samples/PolySpatial/InputMaps and in Assets/Samples/XRInteractionToolkit/3.0.5/VisionOS/Input). Select the XRIPolySpatialInputActions and assign them via button click as project-wide input actions.

The XRIPolySpatialInputActions input actions should look like this:

and

And PolySpatialXRHMD should be added to the Settings panel:

OpenXR settings

Eye Gaze Interaction Profile needs to be added to the OpenXR settings:

The Apple visionOS settings remain unchanged:

Mind it that the setting Upper Limb Visibility currently is not set by the play-to-device mode for unbounded volume cameras, but only is enforced when building (PolySpatial 1.3.9).

PolySpatial settings

unchanged:

Make GameObject interactable

Fix broken shader

If there is a shader errors flagged in line 146, replace this line with #if defined(SHADOWS_SHADOWMASK) && (UNITY_ALLOWED_MRT_COUNT > 4) in the the Unity shader graph core package. File name: PBRDeferredPass.hlsl

Packages

For builds, PolySpatial 1.3.9 requires to reduce the dependency for the collections package to 1.2.3, which also makes the gltf package fall down to an earlier version that is not yet dependent on a higher collections package.

---

OLD:

Alternative script injection

You may also need the XRTouchSpaceInteractor script with the same input action reference.

Buttons can use the simple script VisionOSButtonInteraction to invoke a unity event. The script looks like this:

using System;
using UnityEngine;
using UnityEngine.Events;
using PolySpatial.Samples;

namespace MirageXR
{
    public class VisionOSButtonInteraction : SpatialUI
    {
        public UnityEvent onClick;

        public override void Press(Vector3 position)
        {
            Debug.Log("Button pressed");
            onClick?.Invoke();
        }

    }
}

The script can be assigned like this in one of the spatial view scripts, this is from the MainScreenSpatialView script:

#if UNITY_VISIONOS
        void Start()
        {
                _buttonSidebarCollapse.gameObject.AddComponent<VisionOSButtonInteraction>();
                _buttonSidebarExpand.gameObject.AddComponent<VisionOSButtonInteraction>();
                _buttonSettings.gameObject.AddComponent<VisionOSButtonInteraction>();
                _buttonSorting.gameObject.AddComponent<VisionOSButtonInteraction>();
                _buttonAddNewActivity.gameObject.AddComponent<VisionOSButtonInteraction>();
        }
        public void SetActionOnButtonSidebarCollapseClick(UnityAction action) => _buttonSidebarCollapse.gameObject.GetComponent<VisionOSButtonInteraction>().onClick.AddListener(action);
        public void SetActionOnButtonSidebarExpandClick(UnityAction action) => _buttonSidebarExpand.gameObject.GetComponent<VisionOSButtonInteraction>().onClick.AddListener(action);
        public void SetActionOnButtonSortingClick(UnityAction action) => _buttonSorting.gameObject.GetComponent<VisionOSButtonInteraction>().onClick.AddListener(action);
        public void SetActionOnButtonAddNewActivityClick(UnityAction action) => _buttonAddNewActivity.gameObject.GetComponent<VisionOSButtonInteraction>().onClick.AddListener(action);
        public void SetActionOnInputFieldSearchValueChanged(UnityAction<string> action) => _searchField.SafeSetListener(action);

#else
        public void SetActionOnButtonSidebarCollapseClick(UnityAction action) => _buttonSidebarCollapse.SafeSetListener(action);
        public void SetActionOnButtonSidebarExpandClick(UnityAction action) => _buttonSidebarExpand.SafeSetListener(action);
        public void SetActionOnButtonSortingClick(UnityAction action) => _buttonSorting.SafeSetListener(action);
        public void SetActionOnButtonAddNewActivityClick(UnityAction action) => _buttonAddNewActivity.SafeSetListener(action);
        public void SetActionOnInputFieldSearchValueChanged(UnityAction<string> action) => _searchField.SafeSetListener(action);
#endif

Remove MRTk data providers

Most (all?) data providers can be removed as we phase it out.

MRTK config data providers:

Check branch herman1412/visionos-build-test for working button interaction