Recommended Hardware for Intense Application

[dolai1]

Hello. I am running Frigate 0.13.2 (will migrate to 0.14 with a planned hardware upgrade, unless you think that's a bad idea) in a rather intense application and am wondering what you would recommend for CPU, GPU, and drives. 4 x PCIe Corals will be moved to the new system; they get inference speeds of 7-8 ms normally. I have read the "Recommended Hardware" section, but I think that the circumstances warrant a query.

There are 27 cameras, mostly Dahua/EmpireTech 4k units (Dahua/EmpireTech: 8 x IPC-T58IR-ZE-S3, 6 x IPC-Color4K-T, 3 x IPC-T5842T-ZE, 2 x PTZ5A4K-25X, 2 x PTZ5A4M-25X; Amcrest: 2 x IP4M-1055EW and 1 x IP2M-850EB; Reolink 2 x Video Doorbell PoE and 1 x Video Doorbell WiFi). All cameras are being both recorded and detected.

The current system is an old custom Super Micro server in a 2 U case with 2 x 3.00 GHz Intel Xeon E5-2690 v2s (10-core CPUs), 384 GB of ECC RAM (I think that its usual use is about 15 GB), and no dedicated GPU. The system drive is a 256 GB Samsung 850 Pro MZ-7KE256BW. The storage drives are 4 x 8 TB Western Digital WD80PURZ in RAID 5. It handles the load (CPU utilization about 25% most of the time) and records without a hitch, but uses way too much (electrical) power for the application, not only to run the system, but to cool the room, haha.

So, what direction should I go in designing a new system (CPU, GPU, drives) for Frigate? Your input is appreciated and, I think, of use to others setting up Frigate servers, as well.

[NickM-27]

the main thing that is going to be limiting you is GPU, using the CPU for the decoding will be slower and in general might run in to issues with things happening in a timely manner on the system. As for what GPU, well you'd want one that has enough capacity to decode most if not all of the streams

[dolai1]

Oh, nice. I missed that; thanks.

Do you suggest using the Nvidia card for detection (TensorRT route), or sticking with the Corals (which seems to be working well as-is)?

As far as decoding, just follow the directions at https://docs.frigate.video/configuration/hardware_acceleration/ (driver, Docker reservations/devices/gpu, hwaccel_args)?

[NickM-27]

I would stick with corals if you have them

Yes that's all that is needed.

[dolai1]

Corals: Got it.
Configuration: Nice.

Any special considerations on the CPU, other than that it support VAAPI (mine is old and doesn't)? Is this moot with a discrete GPU?

[pyrodex]

Corals: Got it. Configuration: Nice.

Any special considerations on the CPU, other than that it support VAAPI (mine is old and doesn't)? Is this moot with a discrete GPU?

VAAPI isn't required since your going a discrete GPU IMHO. What is your current CPU usage in the existing system?

[dolai1]

@pyrodex, ~25% most of the time (dual Xeon).

[krk628]

you wouldn't get a discrete GPU if your goal is to reduce power and heat, especially when it's not warranted by your existing performance metrics.

stick w/corals for object detection, avoid discrete gpus, get a cpu w/integrated gpu/vaapi if you're concerned in that area, reduce RAM to no more than 16GB as that consumes power too. as documentation says, use h264 wherever possible for maximum compatibility/user experience. ensure you are using low-resolution streams for detection. post your config.yml w/o passwords if you want more eyeballs on that.

curious...how are you supporting 4 x mini-pci-E corals? your hardware choices will be guided by those to an extent.

[dolai1]

Isn't a modern CPU leaps and bounds more efficient (performance per watt) than my 12-year-old dual Xeon pair? And, at only 25% load on this monster, I could get away with less "horsepower".

I assumed that discrete GPU would be able to throttle back whatever it's not using (since I am chiefly interested in the decoder). Would in integrated GPU have similar decode capacity?

Yes, I'll scale the RAM way back; that's a good point. Any reason to go with ECC in this application (I'm guessing that it's an unnecessarily cost).

H.264: Yes, I have and will.

Detect streams are 720p, except for four at 1080p (they are my default birdseye on a couple 4k displays.

Thanks for the offer on the config.yml. I will redact it and post it when I have a chance (it's huge, haha).

PCIe corals are, very inefficiently, on adapter cards (https://www.amazon.com/gp/product/B0B6HYW2CB), currently wasting four slots. It would be more efficient if I could stack 2 or more per slot, but I never did find a good multi-mini-PCIe adapter card.

[krk628]

more efficient
yes, far less. use your own judgement here as you don't want to be undersized on a CPU, but I normally size cpus on sites like userbenchmark to compare and assess what I need. My current AMD-Ryzen-7-5700U is close in performance to one of your Xeons for example. difference is my TDP is 25W and your Xeon TDP is 95W...so vast difference in power and cooling requirements.

Would in integrated GPU have similar decode capacity?
no, but getting a discrete gpu runs opposite to your stated goals and you haven't made a case for why you'd even need one.

go with ECC
your call. but ECC puts your motherboard/chipset/cpu in a higher class which generally means more power & heat.

Detect streams are 720p
yeah, sometimes 720p is warranted on detect for display purposes. but it's not for detection. going lower reduces performance requirements and reduces load.

currently wasting four slots
sounds like 4 x Corals is also oversized but you'd know better than me. but obviously 4 x slots means a larger motherboard. I use usb corals which are kinda trash compared to mini PCI and m.2 (they tend to overheat). but I deal with it as everything on my system is on a mini PC that fits in my palm.

[dolai1]

@krk628, thank you for the pointers. @NickM-27 also mentioned the role of a GPU in facial recognition, which is something that I'd like to do but am not yet. Is this something that can also be done on the CPU?

I also want to stick with RAID 5, which also changes the class of motherboard that I am looking at.

[krk628]

Is this something that can also be done on the CPU?

yes, depending on the detector you choose. Compreface can do it as long as cpu has avx extensions. to my knowledge there's no native facial recognition planned in frigate. if you look around you'll find that most folks use https://github.com/mincka/double-take to provide integration with face detectors like compreface via MQTT (take a snapshot from frigate, run detection, update the event if appropriate).

personally i'd never install a dedicated gpu for facial recognition as i don't have anywhere near enough volume of events w/persons in my cameras to warrant it. certainly there are many worthwhile applications: any entry/exit (condominium, retail store, employee clock-in/out, etc). 27 cameras is on the larger side as far as frigate goes so that suggests facial recognition might have some value for you. ymmv

I also want to stick with RAID 5, which also changes the class of motherboard that I am looking at.

yup, I get it. i'm NVMe for production....satisfied requirements and is far cooler/quieter. I only use rotational for backups and then I just spin them up daily, do the backups, and power them off.

[dolai1]

@krk628, makes sense.

Spinning rust vs. solid state: That sounds like a really good way to do it. It uses the drive types in a manner that plays to their strengths. Thanks for the idea.