Multiple Nvidia Cards for Hardware Acceleration? #7792
-
I found a good deal on an Nvidia T400 card I ordered to go alongside my existing P600. I currently have 12 cameras balancing between the iGPU and P600 but the discreet card is nearly maxed out with its 2GB of VRAM. The T400 has 4GB so I would like to add instead of replacing the p600 if possible. So according to the below examples from the docs enabling both cards in the Docker Compose file seems straight forward. My question however is how exactly do you assign a particular camera stream for detection to a particular NVIDIA GPU? Its easy to assign / differentiate between the iGP & a single NVIDIA card by adding ether Anyone figured this out? Frigate Docs:
Docker Docs:
|
Beta Was this translation helpful? Give feedback.
Replies: 21 comments 49 replies
-
it's possible to use two separate cards for decoding and object detection, I am not aware of a way to use separate cards for separate streams in frigate or in go2rtc via presets. You can of course do this via manual ffmpeg args |
Beta Was this translation helpful? Give feedback.
-
So I wonder how how stream decoding would be assigned then between two NVIDIA cards in such a configuration. I will cross post this over on the go2rtc forum & then report back here if they chime in with any additional info. |
Beta Was this translation helpful? Give feedback.
-
My T400 card arrived and seems to be working fine / recognized in Docker (GPU 0) but despite the reference Nick posted above I still can't get any streams over to the P600 (GPU 1). Notice the No errors are showing up in either the Frigate or go2rtc logs and and the stream seems to be working fine but just still on GPU 0. At the bottom below I have my updated Docker Compose where I adjusted the devices parameter to now read Any ideeas?
nvidia-smi SMI
Docker Compose file
|
Beta Was this translation helpful? Give feedback.
-
That's also what I suspected at first but the docs over on the go2rtc page don't make much sense to me in that they have a separate wiki page explaining hardware acceleration and all of the # parameters you can string together but as you pointed out you can't just "inject" other ffmpeg arguments by sticking a # in front of it. Then the main page doesn't spell out how exactly you use the #input overall and especially when you are using the examples found on the hw accell page. I cross posted this post over there but never got a response... O well, giving up for now. Suppose I will just pull the P600 and stick with the new T400 since at least I has 2GB more VRAM. |
Beta Was this translation helpful? Give feedback.
-
OK, trying another approach here - this time taking go2rtc out of the mix. Above I was focusing on using the second Nvidia GPU for reencoding my sub-stream that's handles by go2rtc. Why not assign the second GPU to handle Frigate specific function? So farther down in my config where have my global ffmpeg preset I started with this;
Since Frigate doesn't have a preset for pointing to a second Nvidia GPU I tries this;
Unfortunately nothing but vomit in my log... I referred to this Frigate doc for presents for inspiration on how to properly format the above line. |
Beta Was this translation helpful? Give feedback.
-
Odd. I went with your suggestion that less = more and simply used First The second screenshot is run from within the container verifying it can see both GPUs so no idea why the Frigate ffmpeg decoding isn't going to GPU 1. |
Beta Was this translation helpful? Give feedback.
-
One step forward, one step backyards. After adding
|
Beta Was this translation helpful? Give feedback.
-
What you could try is change the order of the devices, for example if you bigger card, is 1, you can swap the order they are presented to the container by
where GPU id 1 is the better faster card, and 0 is the older card that can be used for object detection. |
Beta Was this translation helpful? Give feedback.
-
Hmm, interesting. Didn't know that parameter existed. Hold that thought for a second as I did more testing and found something interesting even with just a single Nvidia card installed. I want to solve / understand this problem first before introducing the second Nvidia card. So in my case I am trying to accomplish multiple things in terms of assigning separate tasks to a particular GPU. Fist (irrespective of Frigate) I am configuring got2rtc to reencode my 12 main camera streams to something workable & uniform (1920x1080 10 fps) and present those as my substream to Frigate for detection purposes. With 12 cameras this would totally overpower the iGPU on top of everything else but the Nvidia cards do this with ease. So far so good. Step 2 is where the trouble starts. One would think that the discreet Nvidia card (single card in this example to make it simple) would also easily handle the Frigate decoding / acceleration also essentially leaving the iGPU idle - doesn't seem to be the case however. In my config if have Even if I disable the iGPU in the BIOS where the OS only sees the Nvidia card I get the same result which seems to downplay the theory of Frigate getting confused with two GPUIs in play. So for now my rig is at least working the way I described above by assigning Frigate specific functions to the iGPU and essentially only utilizing the Nvidia for the go2rtc transcoding (which also benefits from decoding I assume when the stream is being read read in). Although everything is working for now in my case I still want to be able to mix & match for max flexibility. Bottom line is I don't know what's going wrong here as I essentially can't replicate even basic success for using the Nvidia card in the way the official docs specify via the |
Beta Was this translation helpful? Give feedback.
-
That's what I first suspected but as you can see it's only 35% of whatever "GPU %" refers to, Hardware tested = Single Nvidia T-400 (also tried P600 & 1080 Titan) , standard i7-6700 in an HP EliteDesk 800 G2, 6700K in an Asus ROG motherboard, 16GB RAM / 768 dedicated to the container, vanilla Ubuntu Server 22.04 rebuilt from scratch several times. Not sure what I am doing wrong / special. |
Beta Was this translation helpful? Give feedback.
-
Found this command - even with 12 cameras going (2x @4k, 6x @2560,1440, 3x @1920x1080, & 1x @ 2560x1920) max decode = 62% & max encode = 25% |
Beta Was this translation helpful? Give feedback.
-
There is hard limit on the number of streams a NVIDIA GPU device can encode/decode https://developer.nvidia.com/video-encode-and-decode-gpu-support-matrix-new |
Beta Was this translation helpful? Give feedback.
-
The hard limit is for encodes but has been "patched" successfully for ~4 years now. with this patch. Here are my logs with iGPU
Nvidia
|
Beta Was this translation helpful? Give feedback.
-
Ran |
Beta Was this translation helpful? Give feedback.
-
You "should" be able to specify the ~~I dont know if there is a clear advantage between iGPU and NVidia, its on the object detection side, it seems that NVIDA based tensorrt seems to newer models first, like yolov7 etc.. but again this is just based on my experience. ~~ I have multiple Nvidia GPU's but one of them is mining GPU that has encoding/decoding disabled. So I haven't had to target a 2nd GPU for ffmpeg/go2rtc. |
Beta Was this translation helpful? Give feedback.
-
No workie.... Sort of back to where we started. There seems to be various ways to pass what we want with native ffmpeg but the problem is getting it translated into go2rtc speak. Blows my mind to learn go2rtc allows custom presets but now I am thinking everything in between the #s are just that and you could pass virtually any command by defining a preset. We were close above but just ran out of gas / ideas. |
Beta Was this translation helpful? Give feedback.
-
It works!!!!!!! Hang tight while I put my full config back and see if the steering wheel shakes when I hit 80mph. |
Beta Was this translation helpful? Give feedback.
-
makes me want to order in some P600 GPUs ! nice! |
Beta Was this translation helpful? Give feedback.
-
Probably should mentioned this earlier, but if you can setup the record streams 'correctly' on the device itself like the IP camera, I use the the I have setup my camera to the resolution and fps. I want to for the recording. For the detect stream which only read by frigate I have it doing the GPU decode only.
|
Beta Was this translation helpful? Give feedback.
-
One last observation. With one of the Nvidia GPUs + iGPU power consumption for the entire box as about 160w. Now with the addition of a second GPU and the iGPU idle total power draw is down to 111w. Go figure. Adds up for a box running 24x7x365. |
Beta Was this translation helpful? Give feedback.
-
I dont know if this is related or not, but i have an integrated Intel GPU and just recently added an old AMD Radeon HD 6450 i had laying around in a closet to my machine. According to the docs i would need to change the Right now i have my hw_accel_args set to just The two cards in the machine are: |
Beta Was this translation helpful? Give feedback.
Oh copied an older one, try