Multi-Stream Architecture

[HirogaKatageri]

Hi @pedroSG94 I'd like to ask about the Architecture for the Multi-Streaming, this is just to get more of an overview.

In Display Base we have:

• OffScreenGlThread
• MediaProjection - Screen Capture purposes
• MediaProjectManager - Screen Capture purposes
• VideoEncoder - Encoding the Video
• MicrophoneManager - Managing the Microphone
• AudioEncoder - Encoding the Audio
• SurfaceView - Screen Capture purposes
• RecordController - Output the Screen Capture into Device Memory
• FpsListener

I have some questions on what is the use of OffScreenGlThread, and FpsListener?
As well as just to confirm that we only have one VideoEncoder in DisplayBase which outputs the frames into our RTMP Clients right? (One VideoEncoder to Many RtmpClients)

[pedroSG94]

OffScreenGlThread is a OpenGL render that can be used in background. This render get each frame produced by MediaProjectManager, proccess it using OpenGL and send result to VideoEncoder, this allow you a way to apply multiple modifications to the frame like filters.

FpsListener is only a callback that count each frame send to rtmp/rtsp module and report you each second in order to know how many frames are streamed if you haven't internet issues.

Yes, we have only one of each one.

[pedroSG94]

Yes, but to avoid problems related with modify this data we are duplicating it (bytebuffer.duplicate()).

[HirogaKatageri]

Would you recommend having 5 or more RtmpClients doing stuff in the background? A while ago I was testing using 3 RtmpClients streaming and the performance of my phone dropped significantly.

I might play around with the RtmpClient later with Coroutines just to see if it will be of any help.

[pedroSG94]

I didn't performance test yet. I only tested with 3 at same time in 480p but no problems in my case.

Can you tell me device model and video/audio configuration to reproduce your tests?

[HirogaKatageri]

Device: Nokia 7.2
Configuration:

• FPS: 30
• Video Quality: 720p
• DPI: 72
• Video Bitrate: 3 mbps
• Audio Bitrate: 128 kbps

[pedroSG94]

I was doing tests but fps keep constant.
If you mean performance outside of the stream. I think that the only way is share flv packet with all streams but this need a big change.