[BUG]: Starting a VR setup before starting stream breaks stream

[EricXu1728]

Package version

3.1.0-exp.8

Environment

* OS: Windows 11
* Unity version: 2022.3.23f1
* Graphics API: DX12
* Browser: Google Chrome Version 125.0.6422.141 (Official Build) (64-bit)

Steps To Reproduce

1. Download and open Unity VR core
2. Follow the "Creating the Scene" instructions to set up streaming (https://docs.unity3d.com/Packages/com.unity.renderstreaming@3.1/manual/create-scene.html). Put the broadcaster in a separate camera from the VR setup
3. Put on the VR headset BEFORE you hit play on the locally hosted webpage. This should "wake up" the VR set up.

Current Behavior

The stream is black and multiple errors appear in console. I was unable to replicate the issue with normal cameras.

Logs:
Signaling: Connecting WS ws://127.0.0.1
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSCreate () (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/Signaling/WebSocketSignaling.cs:187)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSManage () (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/Signaling/WebSocketSignaling.cs:153)
System.Threading.ThreadHelper:ThreadStart ()

Signaling: WS connected.
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSConnected (object,System.EventArgs) (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/Signaling/WebSocketSignaling.cs:265)
WebSocketSharp.Ext:Emit (System.EventHandler,object,System.EventArgs)
WebSocketSharp.WebSocket:open ()
WebSocketSharp.WebSocket/<>c__DisplayClass201_0:b__0 (System.IAsyncResult)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

Signaling: Receiving message: {"from":"8c07cb5d-fdab-4599-aba9-f906b7506d08","to":"","type":"offer","data":{"sdp":"v=0\r\no=- 7846434483137186184 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=extmap-allow-mixed\r\na=msid-semantic: WMS\r\nm=application 9 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 0.0.0.0\r\na=ice-ufrag:5ybb\r\na=ice-pwd:EDQgn7DfLLwALF/QA4AnH2eU\r\na=ice-options:trickle\r\na=fingerprint:sha-256 78:3C:96:94:13:49:30:B9:05:A8:A2:6E:82:99:5E:72:30:51:C1:9B:C8:FE:51:92:75:2D:4A:F6:C5:C9:CB:8C\r\na=setup:actpass\r\na=mid:0\r\na=sctp-port:5000\r\na=max-message-size:262144\r\n","datetime":1717531019119,"polite":false}}
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/Signaling/WebSocketSignaling.cs:194)
WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:message ()
WebSocketSharp.WebSocket/<>c__DisplayClass174_0:b__1 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:b__3 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:b__0 (byte[])
WebSocketSharp.Ext/<>c__DisplayClass60_0:b__0 (System.IAsyncResult)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

Signaling: Receiving message: {"from":"8c07cb5d-fdab-4599-aba9-f906b7506d08","to":"","type":"candidate","data":{"candidate":"candidate:2823404550 1 udp 2113937151 79e18b59-0f3b-4f02-900b-f37c03ec2013.local 62495 typ host generation 0 ufrag 5ybb network-cost 999","sdpMLineIndex":"0","sdpMid":0,"datetime":1717531019120}}
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/Signaling/WebSocketSignaling.cs:194)
WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:message ()
WebSocketSharp.WebSocket/<>c__DisplayClass174_0:b__1 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:b__3 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:b__0 (byte[])
WebSocketSharp.Ext/<>c__DisplayClass60_0:b__0 (System.IAsyncResult)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

[impolite-PeerConnection ConnectionState:New IceConnectionState:New SignalingState:Stable GatheringState:New] this candidate can't accept on state.
UnityEngine.Logger:Log (UnityEngine.LogType,object)
Unity.RenderStreaming.PeerConnection:OnGotIceCandidate (Unity.WebRTC.RTCIceCandidate) (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/PeerConnection.cs:295)
Unity.RenderStreaming.SignalingManagerInternal:OnIceCandidate (Unity.RenderStreaming.Signaling.ISignaling,Unity.RenderStreaming.CandidateData) (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/SignalingManagerInternal.cs:413)
Unity.RenderStreaming.Signaling.WebSocketSignaling/<>c__DisplayClass39_3:b__4 (object) (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/Signaling/WebSocketSignaling.cs:248)
UnityEngine.UnitySynchronizationContext:ExecuteTasks ()

Signaling: Receiving message: {"from":"8c07cb5d-fdab-4599-aba9-f906b7506d08","to":"","type":"candidate","data":{"candidate":"candidate:1661771167 1 udp 1677729535 71.121.236.94 62495 typ srflx raddr 0.0.0.0 rport 0 generation 0 ufrag 5ybb network-cost 999","sdpMLineIndex":"0","sdpMid":0,"datetime":1717531019162}}
UnityEngine.Logger:Log (object)
Unity.RenderStreaming.Signaling.WebSocketSignaling:WSProcessMessage (object,WebSocketSharp.MessageEventArgs) (at ./Library/PackageCache/com.unity.renderstreaming@3.1.0-exp.7/Runtime/Scripts/Signaling/WebSocketSignaling.cs:194)
WebSocketSharp.Ext:Emit<WebSocketSharp.MessageEventArgs> (System.EventHandler`1<WebSocketSharp.MessageEventArgs>,object,WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:messagec (WebSocketSharp.MessageEventArgs)
WebSocketSharp.WebSocket:message ()
WebSocketSharp.WebSocket/<>c__DisplayClass174_0:b__1 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass84_0:b__3 (WebSocketSharp.WebSocketFrame)
WebSocketSharp.WebSocketFrame/<>c__DisplayClass77_0:b__0 (byte[])
WebSocketSharp.Ext/<>c__DisplayClass60_0:b__0 (System.IAsyncResult)
System.Threading._ThreadPoolWaitCallback:PerformWaitCallback ()

ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
System.Collections.Generic.List1[T].get_Item (System.Int32 index) (at <b11ba2a8fbf24f219f7cc98532a11304>:0) UnityEngine.Experimental.Rendering.XRPass.GetViewport (System.Int32 viewIndex) (at ./Library/PackageCache/com.unity.render-pipelines.core@14.0.10/Runtime/XR/XRPass.cs:195) UnityEngine.Rendering.Universal.Internal.DrawObjectsPass.ExecutePass (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.Internal.DrawObjectsPass+PassData data, UnityEngine.Rendering.Universal.RenderingData& renderingData, System.Boolean yFlip) (at ./Library/PackageCache/com.unity.render-pipelines.universal@14.0.10/Runtime/Passes/DrawObjectsPass.cs:186) UnityEngine.Rendering.Universal.Internal.DrawObjectsPass.Execute (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.RenderingData& renderingData) (at ./Library/PackageCache/com.unity.render-pipelines.universal@14.0.10/Runtime/Passes/DrawObjectsPass.cs:156) UnityEngine.Rendering.Universal.ScriptableRenderer.ExecuteRenderPass (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.ScriptableRenderPass renderPass, UnityEngine.Rendering.Universal.RenderingData& renderingData) (at ./Library/PackageCache/com.unity.render-pipelines.universal@14.0.10/Runtime/ScriptableRenderer.cs:1507) UnityEngine.Rendering.Universal.ScriptableRenderer.ExecuteBlock (System.Int32 blockIndex, UnityEngine.Rendering.Universal.ScriptableRenderer+RenderBlocks& renderBlocks, UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.RenderingData& renderingData, System.Boolean submit) (at ./Library/PackageCache/com.unity.render-pipelines.universal@14.0.10/Runtime/ScriptableRenderer.cs:1463) UnityEngine.Rendering.Universal.ScriptableRenderer.Execute (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.RenderingData& renderingData) (at ./Library/PackageCache/com.unity.render-pipelines.universal@14.0.10/Runtime/ScriptableRenderer.cs:1223) UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderSingleCamera (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Rendering.Universal.CameraData& cameraData) (at ./Library/PackageCache/com.unity.render-pipelines.universal@14.0.10/Runtime/UniversalRenderPipeline.cs:664) UnityEngine.Rendering.Universal.UniversalRenderPipeline.RenderCameraStack (UnityEngine.Rendering.ScriptableRenderContext context, UnityEngine.Camera baseCamera) (at ./Library/PackageCache/com.unity.render-pipelines.universal@14.0.10/Runtime/UniversalRenderPipeline.cs:848) UnityEngine.Rendering.Universal.UniversalRenderPipeline.Render (UnityEngine.Rendering.ScriptableRenderContext renderContext, System.Collections.Generic.List1[T] cameras) (at ./Library/PackageCache/com.unity.render-pipelines.universal@14.0.10/Runtime/UniversalRenderPipeline.cs:370)
UnityEngine.Rendering.RenderPipeline.InternalRender (UnityEngine.Rendering.ScriptableRenderContext context, System.Collections.Generic.List`1[T] cameras) (at :0)
UnityEngine.Rendering.RenderPipelineManager.DoRenderLoop_Internal (UnityEngine.Rendering.RenderPipelineAsset pipe, System.IntPtr loopPtr, UnityEngine.Object renderRequest, Unity.Collections.LowLevel.Unsafe.AtomicSafetyHandle safety) (at :0)

Expected Behavior

Webpage should properly stream camera. This functions normally if you hit play BEFORE putting on the VR headset. Apologies for not being able to track down the source of the issue.

Anything else?

No response

[EricXu1728]

Did more digging. Seems to be an issue with VR not playing well with the existence of multiple cameras. Closing for now.

[EricXu1728]

Woops, turns out this is an issue with Unity Render Streaming. I just messed up reproducing the issue.