Frigate Tip: Improving Live View

[hawkeye217]

The Frigate Tips series is a collection of posts where the Frigate developers give tips on how to best configure, use, and maintain your Frigate installation. We've noticed that a fair number of users asking for support are still using the default live view (jsmpeg) provided by Frigate. So this post's tip aims to help users improve the quality and fluency of the live camera views in Frigate.

The History

Frigate began as a simple tool to detect objects in a camera image. This required running complex math on the image data. It turned out that the algorithms trained to recognize objects didn't require images to be very large or even very clear. Reliable detections could be achieved even with standard definition (SD) quality images from a security camera. And smaller images meant that more of them could be processed in the same amount of time.

Historically and even still by default today, Frigate processes your camera's feed at 5 frames per second and runs those frames through object detection, looking for objects of interest.

Birdseye was born soon after as Frigate's solution to seeing all of your camera's decoded frames at once on a composite, mosaic layout. The benefit was that no significant processing power or memory was required, and no additional connection to the camera was needed.

This solution remains available and is still a viable option for many users today. But for other users, a 5 frames per second, low-quality live view with no audio isn't ideal.

A low-resolution, 720p, 5fps live view. Details are blurry and fuzzy and the stream is not fluent.

The Solution - go2rtc

Beginning with version 0.12, Frigate bundled go2rtc, dubbed as the "ultimate camera streaming application". go2rtc makes it easy to take virtually any camera or stream output and turn it into a standard format that can easily be played in a web browser.

This meant that the live view of your camera in Frigate was no longer constrained to the same 5fps video-only feed that Frigate was decoding for object detection and could now include audio and video at the native resolution and frame rate from your camera. Users running Home Assistant could also see the same improved live views right on their dashboard with the Frigate HASS card.

There are many benefits to using go2rtc with Frigate. It provides:

• Video from your camera at its native frame rate and resolution
• Audio from your camera
• Two-way talk for supported cameras
• Browser features like picture-in-picture

Configuring and Using go2rtc

The Frigate documentation gives a thorough overview on all of the options and caveats to keep in mind when configuring go2rtc with your Frigate installation. There is a step-by-step guide and live view configuration page available, so this tip aims to provide a quick overview and common configuration settings for enhanced live views.

Take this basic Frigate config file:

mqtt:
  enabled: False

detectors:
  coral:
    type: edgetpu
    device: usb

cameras:
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://10.0.10.10:554/rtsp_low
          roles:
            - detect
        - path: rtsp://10.0.10.10:554/rtsp_high
          roles:
            - record
    motion:
      mask:
        - 0,0,0.448,0,0.326,0.058,0.215,0.107,0.099,0.146,0.054,0.204,0,0.228

With this config, Frigate uses a Coral device to detect objects in the front_door camera, using the rtsp_low stream from the camera at 10.0.10.10. The rtsp_high stream is used to save recordings of those detected objects. A motion mask is also configured to cover the camera's timestamp.

As configured, Frigate will use the detect stream for the live view. And again, by default, this is limited to 5fps at 720p.

Enabling Frigate's built-in, bundled version of go2rtc is straightforward. You will need to define a go2rtc: section in your Frigate config file and add a camera to it:

go2rtc:
  streams:
    front_door:
      - rtsp://10.0.10.10:554/rtsp_high

Notice how we've used the same camera name here, front_door, as was already defined in your Frigate config file. This is where the magic happens. If these camera names don't match, Frigate won't use go2rtc for your live view without an additional configuration option.

So, the new, final Frigate configuration file with go2rtc added would look like this:

mqtt:
  enabled: False

detectors:
  coral:
    type: edgetpu
    device: usb
    
go2rtc:
  streams:
    front_door:
      - rtsp://10.0.10.10:554/rtsp_high

cameras:
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://10.0.10.10:554/rtsp_low
          roles:
            - detect
        - path: rtsp://10.0.10.10:554/rtsp_high
          roles:
            - record
    motion:
      mask:
        - 0,0,0.448,0,0.326,0.058,0.215,0.107,0.099,0.146,0.054,0.204,0,0.228

And that's it! This is the simplest, fastest way to configure Frigate to use go2rtc. The Frigate UI will now display more fluent, higher resolution live views.

A high-resolution, 4K, 20fps live view. Details are sharp and crisp. The stream is extremely fluent.

FAQ

1. I tried configuring go2rtc like that and it's not working. Why not?

   • Every camera is different. You may need to use the ffmpeg source and transcode the stream. Browsers require a specific codec to play back streams. See the camera settings recommendations. If you've read the documentation and still can't figure it out, feel free to open up a new support discussion and we'd be happy to help you.

2. Why can't I just increase the fps config option to get a better frame rate for my live view?

   • Using a higher fps value in your detect section will increase the frame rate of Birdseye and the live view, but it will also put additional strain on Frigate's object detector and consume more system resources for little benefit.

3. Why can't I just increase the live view height and quality options in my config to get a better looking live view?

   • Increasing these values only changes height and the quality of the decoded image from your detect stream and does not use the native resolution of your camera.

Official Frigate documentation

Step-by-step go2rtc guide

Live view configuration options

---

If this has been a helpful post, give it an upvote and thumbs up. If you have any questions, leave them down below for us!

-The Frigate dev team (Blake @blakeblackshear, Nick @NickM-27, Josh @hawkeye217)

[tpjanssen]

Why not reference to localhost cameras?

mqtt:
  enabled: False

detectors:
  coral:
    type: edgetpu
    device: usb
    
go2rtc:
  streams:
    front_door_high:
      - rtsp://10.0.10.10:554/rtsp_high
    front_door_low:
      - rtsp://10.0.10.10:554/rtsp_low

cameras:
  front_door:
    ffmpeg:
      inputs:
        - path: rtsp://127.0.0.1:8554/front_door_low
          roles:
            - detect
        - path: rtsp://127.0.0.1:8554/front_door_high
          roles:
            - record
    motion:
      mask:
        - 0,0,0.448,0,0.326,0.058,0.215,0.107,0.099,0.146,0.054,0.204,0,0.228

[tpjanssen]

Maybe I'm the missing the point then... So the point is that the live view is always using the go2rtc stream in case the stream name is the same as the camera name?

[NickM-27]

the point is that we see many users that don't use go2rtc at all. This tip is meant to be a low friction way to explain the benefits and allow users to have an easier path to setting it up.

[hawkeye217]

No. Live view does not use go2rtc by default. The point of the tip is to provide the simplest, most straightforward way to run go2rtc for live view rather than the default jsmpeg. The docs cover additional go2rtc features (reducing connections to the camera, webrtc, etc).

[tpjanssen]

Thanks for pointing that out. Maybe you can also clarify where two-way audio comes in?

[hawkeye217]

The go2rtc docs cover two way audio here: https://github.com/AlexxIT/go2rtc?tab=readme-ov-file#two-way-audio

[Xitee1]

But how can I switch from the low res to the high res stream in the live view?
I'm using go2rtc and have 2 streams per camera (main and sub).
I have configured 2 inputs, record with the main stream and detect with the sub stream.

The recordings are in high quality and with audio while the detection uses the muted lower quality stream.
But why is the live view showing me the low quality stream instead of the high quality one?

[NickM-27]

But why is the live view showing me the low quality stream instead of the high quality one?

likely because of the way you named the streams. See the docs for how to customize which stream is used for live https://docs.frigate.video/configuration/live#setting-stream-for-live-ui

[Xitee1]

Thank you so much!!
Finally it is working!

For some reason I thought that this setting is the friendly name of my camera..

Oh and sound is working too!
I tried sooo many different settings and just coudln't get it to work 🤣