Benchmark Thread

[lysxelapsed]

Since performance is an almost constant topic, I'm starting a proper benchmark thread. I found the benchmark video from rope (not roop, my bad 😆)
I restarted roop-unleashed between models because the VRAM usage builds up. Didn't go higher than 4 threads, because 12GB VRAM are (almost) maxxed out at 4 threads - except for no enhancer and DMDNet.
Here's the video and picture (I hope Jennifer Aniston doesn't mind 😆) I used:

https://github.com/C0untFloyd/roop-unleashed/assets/63816546/d04be678-c69e-4c1e-86a0-e5a5866649a2 <- Video

My specs: i7-13700K, 64GB RAM @5600MHz, RTX3060 12GB
Relevant settings -> Face selection method: Selected face / Processing method: In-Memory-processing Image output format: png / Video codec: hevc_nvenc

Threads	No enhancer	Codeformer	DMDNet	GFPGAN	GPEN	Restoreformer
1	4.64 fps -> 58.18 secs	2.57 fps --> 105.05 secs	3.49 fps -> 77.36 secs	3.37 fps -> 80.11 secs	2.68 fps -> 100.74 secs	2.74 fps ------> 98.5 secs
2	8.71 fps -> 30.99 secs	3.98 fps ---> 67.83 secs	6.13 fps -> 44.04 secs	5.7 fps -> 47.36 secs	4.06 fps -> 66.50 secs	4.24 fps -----> 63.67 secs
3	11.34 fps -> 23.80 secs	4.49 fps ---> 60.13 secs	6.59 fps -> 40.97 secs	6.33 fps -> 42.65 secs	4.33 fps -> 62.35 secs	4.63 fps -----> 58.31 secs
4	12.14 fps -> 22.24 secs	4.59 fps ---> 58.82 secs	7.42 fps -> 36.38 secs	6.83 fps -> 39.53 secs	4.36 fps -> 61.92 secs	4.77 fps -----> 56.60 secs

Here's the code for the 4x7 grid:

| Threads | No enhancer | Codeformer| DMDNet | GFPGAN | GPEN | Restoreformer|
|--------|--------|--------|--------|--------|--------|--------|
| 1 | x fps| x fps| x fps| x fps| x fps| x fps|
| 2 | x fps| x fps| x fps| x fps| x fps| x fps|
| 3 | x fps| x fps| x fps| x fps| x fps| x fps|
| 4 | x fps| x fps| x fps| x fps| x fps| x fps|

[C0untFloyd]

It seems you scared people away with your system specs 😄
And I believe I saw the video on the original roop page as well or perhaps face fusion at one point...
3 more settings affect speed: image output format, video codec and face selection method. I'm using jpg, hevc_nvenc and first found for fastest results. Also extract frames to media should in theory be faster because there's no need to sync the threads. To spare my SSD I used In-Memory anyway. And since we're using the same video, shouldn't we measure running time too?

I5-13600K, 32GB RAM @4400MHz, RTX2060 SUPER 8GB

Threads	No enhancer	Codeformer	DMDNet	GFPGAN	GPEN	Restoreformer
1	4.29 fps -> 63.07 secs	x fps	x fps	x fps	x fps	x fps
2	9.38 fps -> 28.90 secs	x fps	x fps	x fps	x fps	x fps
3	9.89 fps -> 27.43 secs	x fps	x fps	x fps	x fps	x fps
4	12.75 fps -> 21.29 secs	x fps	x fps	x fps	x fps	x fps

Very inconsistent results, can't explain the small jump between 2 & 3 and the big jump once I used 4 threads.
Might update this with more results over time.

[lysxelapsed]

😆 Yeah the bottleneck is definitely the GPU in my case 😄 Gave my old 4770k-rig a well-deserved upgrade last year, will probably last 7-8 years again 😁 But I expected this somehow.. everyone asks for a benchmark to see if they're in the ballpark. But doing it yourself? Hmmm 🤣
I noticed some inconsistencies too when doing these test. I see mainly 2 reasons for that: 1. The short video. The first 10-15% (and that's a lot for a benchmark) are not processed at full speed and some enhancers take a bit longer to "gear up" than others, and even with no enhancer you won't get the exact same result twice. I considered at least doubling the video lenght to smooth that out a bit, but that would scare even more people away from doing it. And 2. The "VRAM-(non)-Offloading". VRAM still doesn't get cleared completely after processing, even though it's a lot better than in previous versions. I only restarted completely between enhancers and went from 1 thread one by one to 4 thread. If you want a true comparison, you'd have to restart completely for each run, or at least start with 4 (or whatever your max is) and go down one by one. But yeah, your case remains somewhat strange ^^
Ahhh glad you cleared that up with the image output format. I always wondered if it's just the output format of a single image-format-swap, or if it applies to the frames extracted from a video - and yeah, was too lazy to test it with extracting the frames 😆
The In-Memory-Swapping is the biggest upgrade to this day in my opinion. I'd never go back to wearing the SSD (or even HDD) down, I like those running for lots of years 😉
Oh and I'll update the post with times and settings - I stayed on selected face since it's probably the most used method.

[C0untFloyd]

https://github.com/facefusion/facefusion-assets?tab=readme-ov-file#examples

Seems like we all borrow from each other (the example videos) 😄

[C0untFloyd]

I did some optimization experiments and removed the genderage model from the workflow (almost never use the female/male option). With 1 thread the video processing took only half the time ~ 32 secs. Funnily doing it again with 4 threads it was a meager 1 sec difference to before (I didn't restart though). Perhaps writing to disc blocks the process...

[lysxelapsed]

Interessting. In general, and as you can see in my benchmark, even going from 2 to 3 threads doesn't improve performance that much. Even though the VRAM is not maxxed out. And with 3 threads, fps are a lot more "jumpy" than with 2 threads. Probably because the threads need to "sync up" again? Happy to test it out with the removed model, if you decide to push that into the main branch.

[lysxelapsed]

New Benchmark with 3.5.5
There is a small performance improvement. Will be more noticable on longer videos, I'd say it's around 5%. Thx for the update 😃
Codeformer at 4 threads produced a "distorted face square" result -> Probably too many threads / VRAM maxxed out.

System specs: i7-13700K, 64GB RAM @5600MHz, RTX3060 12GB
Relevant settings -> Face selection method: Selected face / Processing method: In-Memory-processing / Image output format: png / Video codec: hevc_nvenc / Other settings like similarity threshold and blend ratio at default value
Windows users: Keep the cmd output window active or at least visible.

Threads	No enhancer	Codeformer	DMDNet	GFPGAN	GPEN	Restoreformer++
1	4.76 fps / 57.04 secs	2.63 fps / 102.70 secs	3.58 fps / 75.53 secs	3.45 fps / 78.42 secs	2.70 fps / 100.19 secs	2.77 fps / 97.50 secs
2	8.50 fps / 30.32 secs	4.07 fps / 66.42 secs	6.23 fps / 43.45 secs	5.89 fps / 45.98 secs	4.13 fps / 65.52 secs	4.38 fps / 61.74 secs
3	11.71 fps / 23.20 secs	4.55 fps / 59.41 secs	6.97 fps / 38.90 secs	6.82 fps / 39.74 secs	4.36 fps / 62.07 secs	4.75 fps / 57.03 secs
4	12.77 fps / 21.28 secs	4.58 fps / 59.12 secs	7.56 fps / 38.86 secs	7.28 fps / 37.23 secs	4.40 fps / 61.02 secs	4.80 fps / 56.34 secs

[C0untFloyd]

Remember, this only speeds up face recognition not the swapping itself. A video with multiple faces would see a HUGE speed increase.

[kiriku99]

I tried 20, 16, 12, 8, 6, 4 and 3

3 works best for me. anything above seems to do bursts of speed and then slowdowns. 3 is steady
RTX4080 and i9

[dallas4u]

My bench... surprised me with a few things. Maybe it's because the VRAM isn't offloading completely, but using Restoreformer, I ran it twice for each thread count. The first run was ALWAYS slow. Like three times as slow. The second run, though, is what you see in my chart. So, running it a second time went 3x as fast. Not sure why. Also, using Restoreformer, the second run with 2 threads was faster than the run with 3 threads? I dunno, but that's what I got. Same with GPEN and 2 threads. Maybe due to below...

Also, my setup definitely doesn't like using 4 threads. I didn't completely close down Roop until 4 threads and Codeformer, where it was completely maxing VRAM and GPU and said it was going to take around 25 min! Closing and re-starting wiped VRAM and gave me better results, but I keep it on GFPGAN and 3 threads 99% of the time anyway. Same thing on GPEN, it's holding between 5GB - 6GB VRAM before starting, and GPEN was gonna take around 4.30 min until I restarted Roop. Meaning, all of my tests could just be skewed due to VRAM not releasing whatever it is holding between changing post-processors.

Ryzen 7 7800X3D. 64GB RAM@6000MHz, RTX 4070 Super 12GB

Threads	No enhancer	Codeformer	DMDNet	GFPGAN	GPEN	Restoreformer
1	7.82 fps / 34 secs	4.55 fps / 59 secs	5.63 fps / 47 secs	5.64 fps / 47 secs	4.98 fps / 54 secs	4.61 fps / 58 secs
2	13.17 fps / 20 secs	7.68 fps / 35 secs	8.00 fps / 33 secs	8.97 fps / 30 secs	9.15 fps / 29 secs	7.78 fps / 34 secs
3	15.95 fps / 16 secs	8.15 fps / 33 secs	10.69 fps / 25 secs	11.42 fps / 23 secs	8.03 fps / 33 secs	6.65 fps / 40 secs
4	20.91 fps / 12 secs	5.76 fps / 46 secs	10.73 fps / 25 secs	12.08 fps / 22 secs	8.40 fps / 32 secs	8.84 fps / 30 secs

[htmlcodepreview]

This is very strange. Restoreformer always takes x2 more time for me compared to GPEN.

[dallas4u]

Yeah, I think if I had (or you do) close the app and re-open when testing (or using) a pre-processor (or changing pre-processors), and even run a second time on GPEN and Restoreformer if it seems really really slow, you will get different (faster) results. That was my take away from the testing. But that is a lot of time and effort. That's why I either don't use a pre-processor and upscale/process with something else, or just use GFPGAN when needed.