In this fork of the original ControlNet repo, we evaluated the possibility of using lighter backbones for ControlNet model instead of Stable Diffusion encoder. Our experiments and findings are described here: https://medium.com/@dzvinkayarish/controlnetlite-smaller-and-faster-controlnet-55b26cef946e . The project was inspired by this discussion lllyasviel#188 .