Skip to content
This repository has been archived by the owner on Mar 22, 2022. It is now read-only.

Different behaviour between UnityEditor and Hololens: VideoSource stop to send video if other peer want send video #828

Open
Qualcuno opened this issue Oct 18, 2021 · 0 comments

Comments

@Qualcuno
Copy link

I developed a communication with Hololens 2 and a webrtc webpage ( passing through an SFU ) and i have 2 operative mode:
Hololens ( or UnityEditor ) always offer an audio and a video communication
and Web can
(A) offer audio and video communication
(B) offer just audio communication ( no video needed ).

When i try to start a communication with Unity Editor and the Web, all works fine ( both with (A) and (B) mode works well )
If i try to start a communication with Hololens, something wrong if web want to send video ( (A) mode ) but works well if web not send video ( (B) mode).

In the details, if the web want to send video to Hololens, Hololens not start or stop to send video to web.
But this does not appen in Unity Editor! In Editor the same code works well!.

I manipulate the Hololens frame before put it into stream and i saw that
no OnIncomingI420Frame or OnIncomingARGB32Frame is called.

I checked if any transceivers or tracks were not properly created with this code

private void Update()
{
if (peer != null && peer.Peer != null)
{

        string info = "INFO";

        for (int i = 0; i < peer.Peer.Transceivers.Count; i++)
        {
            Microsoft.MixedReality.WebRTC.Transceiver t = upeer.Peer.Transceivers[i];
            info += "\nTransceiver " + t.MediaKind + " [" + t.MlineIndex + "] " + t.StreamIDs[0] + " " + t.DesiredDirection + " -> " + t.NegotiatedDirection; 
        }

        for (int i = 0; i < peer.Peer.Transceivers.Count; i++)
        {
            Microsoft.MixedReality.WebRTC.Transceiver t = peer.Peer.Transceivers[i];
            if (t.LocalVideoTrack != null)
            {
                info+= "\nLocalVideoTrack " + t.LocalVideoTrack.Name + " enabled: " + t.LocalVideoTrack.Enabled;
            }
        }
       Debug.Log(info);
    }
}

but the result is the same both UnityEditor and Hololens

Transcevier Kind: Audio, mlineindex[0], streamId: mixedmslabel, Desired: SendReceiver Negotiated: SendRecevie
Transcevier Kind: Video, mlineindex[1], streamId: mixedmslabel, Desired: SendReceiver Negotiated: SendRecevie
Transcevier Kind: Audio, mlineindex[3], streamId: ....random_Stream_id_1...., Desired: SendReceiver Negotiated: ReceiveOnly
Transcevier Kind: Video, mlineindex[4], streamId: ....random_Stream_id_2...., Desired: SendReceiver Negotiated: ReceiveOnly
LocalVideoTrack ...random_Track_id_1... enabled: True
LocalVideoTrack ...random_Track_id_2... enabled: True

( where mixedmslabel is an empty stream presents into SFU and MLineIndex 2 is not presents becouse is the datachannel).
I don't know why the MLineIndex 3 and 4 are ReceiveOnly but this output is the same of UnityEditor where web side correctly receive video.

Why VideoSource stop to send video?

Unity 2020.1.17f1
WebRTC 2.0.2

@Qualcuno Qualcuno changed the title Different behaviour from UnityEditor and Hololens: VideoSource stop to send video if other peer want send video Different behaviour between UnityEditor and Hololens: VideoSource stop to send video if other peer want send video Oct 19, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant