Using the MRTK for Touch Input

[gisornator]

Hey,

I didn't want to crash the Hit-Test Issue with that question, since we got that covered for Android and it is working now. However, I would like to have some description on how to properly use the MRTK as input manager to interact with Unity UI elements during an XRSession.

I thought, that I had the MRTK set up properly but I'm recieving a warning now, which suggests, that something is not configured correctly in my project:

[MRTK Issue] Touch 0 has already been registered with the Input Manager!

This is my MRTK config GameObject:

[Rufus31415]

Input sources are set up in "Input Data Provider" expandable menu that we see on your screenshot.
You should add the one provided with SimpleWebXR. You could also disable "Unity Touch Device Manager" It will probably remove your warning, but I don't understand why you get it without seeing the whole project

[gisornator]

We're getting closer I think... I removed the Unity Touch Device Manager and it removed the warning. But I noticed that I already had your Input provider set.

[Rufus31415]

So it should work. A touch on the screen in immersive session should raise a select event :

Simple-WebXR-Unity/com.rufus31415.simplewebxr/Runtime/Plugins/WebGL/SimpleWebXR.jslib

Line 603 in a906041

session.addEventListener('select', _onSessionEvent);

Then the _onSessionEvent handler is called and writes _controllerEvents array :

Simple-WebXR-Unity/com.rufus31415.simplewebxr/Runtime/Plugins/WebGL/SimpleWebXR.jslib

Line 507 in a906041

_controllerEvents[id] = eventTypeMask;

At next frame, the event is propagated to _byteArray :

Simple-WebXR-Unity/com.rufus31415.simplewebxr/Runtime/Plugins/WebGL/SimpleWebXR.jslib

Line 419 in a906041

_byteArray[1] |= _controllerEvents[0];

_byteArray is shared with C# code :

Simple-WebXR-Unity/com.rufus31415.simplewebxr/Runtime/Plugins/WebGL/SimpleWebXR.jslib

Line 129 in a906041

_byteArray = new Uint8Array(buffer, byteArray, byteArrayLength);

At next frame, C# code set Selected property :

Simple-WebXR-Unity/com.rufus31415.simplewebxr/Runtime/Scripts/SimpleWebXR.cs

Line 712 in a906041

inputSource.Selected = true;

--> But maybe here is the issue : your smartphone only raises event Select and not SelectStart so selected is never set to true.

In MRTK, this Selected property is used :

Simple-WebXR-Unity/SimpleWebXR-Demo/Assets/SimpleWebXR/Scripts/MRTK/SimpleWebXRController.cs

Line 70 in a906041

Interactions[i].BoolData = controller.Selected;

[gisornator]

Hmm... weird. Actually, I cannot query for "Selected" because this will always be false. However, I can listen to SelectStart and SelectEnd. So your post makes sense. But for some reason I see Select events when remote debugging on chrome:

[gisornator]

Okay, classic PICNIC issue here. So I probably didn't understand it correctly in the first place but all UI elements have to be world-canvases and they have to be configured for MRTK, otherwise the other elements like buttons etc. aren't interactable. The buttons themselves, however, can be standard Unity UI Buttons - as long as the canvas is configured for MRTK! :)

Anyway, I appreciate your help and I love this repo so far. It makes designing AR apps for WebGL much more convenient as we are not bound to certain 3D object types but can also use .FBX etc.

[Rufus31415]

Thanks for your kind comment ;) Do not hesitate to contribute to SimpleWebXR if you have time and good ideas !