Best practices for remote frigate instance integration into Home Assistant

[igiannakas]

I currently have a remote (off premise) frigate instance that publishes events to my on premise Home assistant instance.

I am running an MQTT Broker setup as a bridge (mosquito) on the remote server where frigate is running that subscribes and forwards the frigate message topics to my on premise Frigate instance. Site to site connectivity is handled by a Tailscale tunnel. This works exceedingly well when network is available.

However when I have network outage on the remote site, it appears that MQTT messages from frigate are lost and this leaves my local HA instance out of sync with the remote events.

I believe this happens because frigate publishes to the MQTT bridge its messages as QOS 0, which means message delivery is at most once, instead of QOS 2 which is exactly once. This means that while the bridge is available during the outage, as it cannot re-write the QOS of the messages, these are attempted to be sent and then dumped, if the destination MQTT server is unreachable.

I have tried to work around this QOS limitation by disabling clean sessions (which retains the session information between bridge and master) and requested my bridge to queue QOS0 messages for retransmission.

The MQTT bridge config is below:

persistence true
persistence_location /mosquitto/data/
persistent_client_expiration 7d
log_dest file /mosquitto/log/mosquitto.log
log_dest stdout
log_type error
log_type warning
log_type notice
log_type information
listener 1883
allow_anonymous true
autosave_interval 1800
autosave_on_changes true

connection BRIDGE-NAME
address [IP]3:1883
bridge_protocol_version mqttv50

cleansession false
keepalive_interval 20
restart_timeout 10
notifications true
notification_topic bridge/status
try_private false
start_type automatic

queue_qos0_messages true
max_queued_messages 2000

topic frigate/# both 0 "" "" "retain as published"

However that has gotten me thinking:

1. Is it possible to enable QOS 2 on the frigate MQTT messages? Receiving the frigate messages in the right order and all of them is important for the HA plug in to maintain presence status etc.
2. Would a "better" way be to connect frigate directly to the remote MQTT instance? What is frigate's behaviour if the MQTT server is temporarily unavailable?

What I am trying to avoid is "fantom" events where for example presence will still show in HA in an area if there was a person present before the outage but left during the network outage. This happens because HA doesnt get the "left" event hence its status is not updated and the only way out is to either wait for another detection or restart frigate.

Also on some instances I have noticed that frigate presence would not update in HA even though there is a person and the bridge is alive and communicating with the master location. The way out there is to reload the HA frigate integration and restart frigate.

Appreciate your thoughts!

[NickM-27]

We could add support for a config that would set the qos that should be used, in general though you could likely just connect them directly I would think, if a disconnect happens it will just loop trying to reconnect

[igiannakas]

I did try that but the remote site upload speed is the constraint. Constant video streaming is intensive and while it was more reliable for 1-2 cameras when I hit 15 or so that I have now it’s more problematic.

Plus im loosing out on event recording when the network is down (loosing notifications for the down period is fine because then I can review the alert recordings anyway). I had that mitigated by running a dual instance - one remote and one on site linked to the go2rtc streams but then I hit the bandwidth limitation again.

while with frigate running remotely and just pushing mqtt messages it’s far more lightweight and also I can view just the camera in question vs stream all of them constantly if that makes sense.

How would frigate react if I connected it to the remote mqtt server (the master in this instance and not the bridge) if the connection was down? Probably a bad idea due to latency etc but just checking :)

[igiannakas]

Am I also missing a trick with the bridge setup perhaps? I’m assuming setting the below up even with qos 0 messages it should cache them and play them back when the destination mqtt is back online?

queue_qos0_messages true
max_queued_messages 2000

[igiannakas]

OK after digging a bit more in mosquito config, I think that it has a non standard option to upgrade the QOS of messages sent and received and also setting the bridge to transmit one message at a time while setting up a large queue should provide some protection against out of order message delivery. So probably the below will work.

In any case having QOS 2 is probably a better idea especially as we rely on message sequencing and all of them being delivered. While local delivery isn't affected really by the downrated QOS, it shouldn't hurt upgrading it or giving an option to do so in the future :)

# Global Settings
persistence true
persistence_location /mosquitto/data/
persistent_client_expiration 7d

# Logging
log_dest file /mosquitto/log/mosquitto.log
log_dest stdout

log_type error
log_type warning
log_type notice
log_type information

# Listener Settings
listener 1883
allow_anonymous true

# Autosave Settings
autosave_interval 1800
autosave_on_changes true

# Upgrade Outgoing QoS
upgrade_outgoing_qos true

# Maximum In-flight and Queued Messages
max_inflight_messages 1
max_queued_messages 10000
queue_qos0_messages true

# Bridge Configuration
connection BRIDGENAME
address [IP]:1883
bridge_protocol_version mqttv50

# Session Settings
cleansession false

# Keepalive and Restart
keepalive_interval 20
restart_timeout 5 30

# Notifications
notifications true
notification_topic bridge/status

# Bridge Behavior
try_private true
start_type automatic

# Topic Configuration
topic frigate/# both 2 "" "" "retain as published"