Control lips, teeth and tongue for lip sync task.

[tumurzakov]

I want precisely control mouth for stylize video. There is model for controlnet ControlNetMediaPipeFace from CrucibleAI. But there is no enought control for lips, teeth and tongue.

1. Is there a way to make lips sync without training special model for control net?
2. Is it possible to train extension for ControlNetMediaPipeFace to make lips sync?

Discussion on huggingface
Redit post

[geroldmeisinger]

ControlNet 1.1 has support for faces now https://github.com/lllyasviel/ControlNet-v1-1-nightly#controlnet-11-openpose

[johndpope]

@tumurzakov - did you use the ControlNet 1.1 implementation? how did you find it? i just spent an hour wiring up the ControlNetMediaPipeFace