v3.2.0 Release

[C0untFloyd]

With the release of v3.2.0 there's a new Tab and a improved way of swapping. I'm calling the files facesets and it's just multiple reference images for a person (or several persons if you want to mix appearances) put into 1 (zip)-file.
Those facesets are used to increase the likeness and quality of the deepfakes and should either be high quality images of the person you want to fake or different persons when you want to mix their faces for whatever crazy reason. It's also a good idea to have different angles of the faces. The faceset files can be used in the input filebox of the face swapper tab, similar to regular images. Only the first thumbnail will be shown in the gallery of faces, but all of the ones in your faceset will be used.

For creating them there is a new Tab page:

The left filebox accepts already existing .fsz files (for modifying them later), the right one accepts images in jpg, png or webp format.
After loading images into the right filebox the faces inside them will be added to the thumbnail gallery below. You can then decide to keep or remove it. When you're done, clicking on the create/update button will create the file.
Dropping an already existing faceset in the left panel will open all the faces inside it. You can then remove or add faces like you did before.

PS: DMDNet Generic will automatically be used when the input is a faceset with several ref faces and you selected DMDNet as enhancer.

Update News to Update 3.3.3 🥳

There is a new dropdown box which I think is self-explaining but just in case, it specifies what to do with the current processed frame when there was no face detected:

• Use untouched original frame -> same as it always was, save frame and continue
• Retry rotated -> Flips the image and tries detecting upside down faces, flips back to original, saves and continues
• Skip Frame -> Totally skips this frame if there was no face detected

Skipping frames is currently only supported for the In-Memory processing, I'm unsure if it makes sense when the frames can be edited by hand anyway.

[ShrekLovesAll]

Insane how far this build has come. Amazing stuff as always countfloyd. Is there a chance we could get a somewhat dumbed down explanation of how DMDNet works/differs from existing enhancers?

[C0untFloyd]

Thanks, but I didn't find out about this and I'm only on 4th place to implement this 😭
But other than the other roop forks I want to build on this and have source/destination presets.
https://github.com/csxmli2016/DMDNet is the project page of DMDNet with some explanations. I'm using the supplied faces to create the needed face landmarks for it. In theory this should again improve the face identity, practically it's a lot slower and I can't spot a difference but I always wanted to finally implement it.

[ShrekLovesAll]

Lol well I'm happy you implemented it anyways. Big fan of the incremental improvements sources from multiple projects. Also, I mean to actually ask about GPEN, not the existing enhancer. Just got confused when I read DMDnet here. DMD has always looked really clear to me but it seems a little aggressive with the correction sometimes, and tends to leave a small ghost box around heads sometimes.

[C0untFloyd]

I believe the box is a thing of the past and yes for DMDNet I decrease the blend ratio to 0.4 else it's too visible.
GPEN is this https://github.com/yangxy/GPEN
and I included it because several other older faceswap apps did use this and it was real easy to add. It's a very fast model, usefull if you want quick results in mediocre to good quality.

[eniora]

WOW this is one of the best features to roop unleashed in a while. Will have lots of fun mixing some faces too 😄
Count, seriously you're so innovative.

[gabrielpimp]

This is brilliant! Thank you so much for your work! If I am so bold as to give you a suggestion, a simple mechanic that would be really helpful would be some way to rotate the target face, so we can help the recognition alg to find the face.

[C0untFloyd]

Yes this is something I also pondered about. It's very easy to do for 1 face but tricky for multiple. Perhaps this should an option for selected faces only. Also it should be limited to video ranges I think, as it slows down the whole process a lot.

[pablo-888]

hell yeah!

how many photos per camera angle would you (or anyone else that already tested this) recommend to get the best results possible? is there diminishing in returns after a certain point?.

[C0untFloyd]

I'm still experimenting myself. The best results I had with about 4-6 images with different angles (side-view, slighty from below or top, several HQ Images from the front). You don't need that many to get better results, but they should have a decent resolution.

[escamez]

Hi there,

Thank you so much for such a great interface, I'm testing it in MacOS and a Debian LXC with passthrough, and so far it works pretty well. I'm trying facesets in Debian only 'cause there is an error in MacOS since it can not create /temp directory. I'm checking the sources myself to see if I can change directory location, what you think about an .env file to set certain vars at start?

Once more, thanks.

[sakeRadio]

fyi C0untFloyd helped me have the facesets running on Mac for now, these 2 alterations fix your /temp issue
#292 (reply in thread)

[C0untFloyd]

Sounds like a user rights problem either in MacOS or the Mac Browsers. I'm sort of (ab)using the env vars, see here:
https://github.com/C0untFloyd/roop-unleashed/blob/main/ui/main.py#L24

You could try commenting out this line and the following 2 by putting a # in front of the lines. It will then use the system temp without creating folders. It could be that I'm using the absolute path to roop's temp later and I'm not at home right now but for a start, you could check if this at least partially works.

[metal3d]

there is the same problem on Linux (Fedora 38), the directory is set to bad mask

ls -la temp/faceset/
total 0
d????????? ? ? ? ?              ? .
d????????? ? ? ? ?              ? ..

I tried to remove the directory and change umask in utils.py file, this doesn't fix the problem. Is it related or should I create an issue ?

[dartusf123]

Really great work! This has been a very welcome addition.

Is there any chance to be able to see the combined face that's created from a faceset? For example, if I'm combining multiple faces or are adding multiple of the same face, it'd be nice to compare/contrast what adding a specific image to the faceset contributes to the combined face. Could also help if I've inadvertently added a picture that actually takes away from the intended result, allowing an easier way to identify the offending image.

Of course this could also be done directly within the face swap tab by cycling through a few variations, but can be a tad time consuming and also may not provide exact details if the face shape that's being swapped doesn't match perfectly.

[lysxelapsed]

+1 for viewable faceset results 😃

[marlonio]

Incredible features @C0untFloyd! 👍

[hansvelas]

What resolution would you consider high quality? Would there be a difference if the resolution were huge?