Slow Detector Inference Speed for Coral TPU on Home Assistant OS running as VM on Qnap NAS (TS-364)

[thesearge]

Hello,

I'am running HA OS on my Qnap NAS on Virtualization Station. I am using a Coral TPU (USB) In order to enhance performance for frigate ( 2 Reolink WLAN-Cams (CX-410W)). Coral TPU is detected by default by Qnap QTS 5.1.8.2823. No need for driver installation at this point. I am able to hand over Coral USB-Device to HA OS at VM. This works, but it is not very stable. Rebooting VM or NAS sometimes ends up not finding Coral TPU in HA OS or VM. In this case I need to bind USB-device again manually and restart frigate add-on at HA OS. Despite this I recognised that Inference Speed for Coral TPU is pretty slow, sometimes slower than 300ms. Running good means approx. 30ms which already seems to me pretty slow compared to other statements for coral TPU enhancements.
I think driver is not running properly. Any help appreciated.

[NickM-27]

VMs are known to cause slowdowns because of the heavy virtualization layers involved

[thesearge]

Sounds reasonable, but a lot of people runnibg slower setups with VMs (Proxmox etc.) and they report much better speeds (<10ms) and good system stability. There must be another reason…