Semantic search and mqtt

[xavan83]

Semantic search is working great and shows impressive results.
I can not see the generated description in the mqtt events.
How can I retrieve it?

[hawkeye217]

I've implemented an mqtt topic, event_update, that will publish a tracked object ID and description when your AI provider generates a description. #13930

[divemasterjm]

i delete the directory but when i deploy it creates automaticaly
now i have this mappings
- /etc/localtime:/etc/localtime:ro
- /root/frigate:/config:rw
- /media/frigate:/media/frigate
- /root/frigate/db:/db
- /root/frigate:/dev/shm/logs

[divemasterjm]

the content of /root/frigate
root@frigate:~/frigate# ls -al
total 4268
drwxrwxrwx 9 root root 4096 Sep 24 17:41 .
drwx------ 10 root root 4096 Sep 24 17:19 ..
-rwxrwxrwx 1 root root 17 Sep 24 17:13 .exports
-rwxrwxrwx 1 root root 128 Sep 24 17:13 .jwt_secret
-rwxrwxrwx 1 root root 17 Sep 24 17:13 .timeline
-rwxrwxrwx 1 root root 16 Sep 24 17:13 .vacuum
drwxr-sr-x 2 nobody nogroup 4096 Sep 7 11:10 certsync
drwxr-sr-x 2 nobody nogroup 4096 Sep 24 17:37 chroma
-rwxrwxrwx 1 root root 25767 Sep 5 08:13 config.05092024
-rwxrwxrwx 1 root root 25767 Sep 6 07:11 config.06092024
-rwxrwxrwx 1 root root 25767 Sep 8 07:42 config.08092024
-rwxrwxrwx 1 root root 21327 Jun 10 16:34 config.100624
-rwxrwxrwx 1 root root 23298 Jul 13 09:05 config.bueno
-rwxrwxrwx 1 root root 24750 Jul 16 19:29 config.buenosingo2rtc.yml
-rwxrwxrwx 1 root root 26693 Jul 18 07:06 config.congo2rtc
-rwxrwxrwx 1 root root 25512 Jul 18 06:49 config.singo2rtc
-rwxrwxrwx 1 root root 26943 Sep 22 14:11 config.yml
-rwxrwxrwx 1 root root 21509 Feb 7 2023 config1024.yml
-rwxrwxrwx 1 root root 21600 Feb 9 2023 config640.yml
-rwxrwxrwx 1 root root 20353 Feb 1 2023 configbeta3.yml
-rwxrwxrwx 1 root root 20924 Feb 1 2023 configbeta4.yml
-rwxrwxrwx 1 root root 21253 Feb 1 2023 configbeta5.yml
-rwxrwxrwx 1 root root 25441 Jul 17 19:30 configbuenamixgo2rtc.yml
-rwxrwxrwx 1 root root 21506 Feb 8 2023 confighd.yml
-rwxrwxrwx 1 root root 18443 Feb 9 2023 configmodhd.yml
drwxrwxrwx 2 root root 4096 Sep 24 17:37 db
-rwxrwxrwx 1 root root 1405 Jan 13 2023 docker-compose.yml
drwxr-sr-x 2 nobody nogroup 4096 Sep 24 17:37 frigate
-rwxrwxrwx 1 root root 114 Sep 24 17:32 frigate-go2rtc.yaml
drwxrwxrwx 2 root root 4096 Jun 28 09:13 frigate-notify
-rwxrwxrwx 1 root root 3861977 Jun 3 22:23 frigate-notify_0.3.0_linux_amd64.tar.gz
drwxrwxrwx 2 root root 4096 Sep 24 17:13 model_cache
drwxr-sr-x 2 nobody nogroup 4096 Sep 23 19:54 nginx
-rwxrwxrwx 1 root root 241 Sep 24 17:13 notifications.pem

[xavan83]

I've implemented an mqtt topic, event_update, that will publish a tracked object ID and description when your AI provider generates a description. #13930

I have the regenerate button next to the Save button (however these button are not visible at first load because the description textarea is too big, a scrollbar appears)

The regenerate button leads to a 404error in chrome debugger

[NickM-27]

because you have

• /root/frigate:/dev/shm/logs

probably better to do

• /root/frigate/logs:/dev/shm/logs

[divemasterjm]

correct, thanks, sorry for my mistake

[divemasterjm]

when i click button to regenerate if got "failed to call gemini for a new description"

[NickM-27]

fixed in #13937

[xavan83]

fixed in #13937

Thank you it is working for 2-3 events. After that I receive a message indicates that it can take some time but nothing appears after several minutes.
Which logger can I activate to débug? Like quota issue may be?

[hawkeye217]

I've put up some additional fixes here #13940

[xavan83]

I've put up some additional fixes here #13940

Still no luck

[hawkeye217]

What AI provider are you using? You could be running up against a quota

[xavan83]

frigate | 2024-09-25 07:22:25.998281479 192.168.0.9 - - [25/Sep/2024:07:22:25 +0200] "PUT /api/events/1727197426.506687-d8ktqg/description/regenerate HTTP/1.1" 200 104 "http://192.168.0.16:5000/explore" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" "-" frigate | 2024-09-25 07:22:26.258377453 [2024-09-25 07:22:26] frigate.genai.gemini WARNING : Gemini returned an error: 429 Resource has been exhausted (e.g. check quota).

You are right
According to the documentation the limit is 15queries per minute (but that is for the flash model, not the pro I was using)

I made some tests regarding the search in the description. One of the car was label as a Renault Kangoo in the description (that is correct). I try to search with Renault Kangoo but the car does not appears in the result.

Do you plan to enhance this feature like doing calls only for specific region and object type?

[NickM-27]

license plates are attributes of car, you can restrict car to a specific zone and specify license plates for that zone only

[xavan83]

Like this:

Would it be possible to combine such a search with clip (like for now) + a crop of a license_plate snapshot(instead of a full snapshot)?

[NickM-27]

you can't apply a license plate filter here because again the license plate does not get attached to the zone

[xavan83]

So how to do "and specify license plates for that zone only" ?

[NickM-27]

add a prompt for that zone and make it be "and specify the license plate number if there is one visible"