KnobGen for Condition Diffusion Tasks (Pouyan Navard*, Amin Karimi Monsefi*, Mengxi Zhou, Wei-Lun (Harry) Chao, Alper Yilmaz, Rajiv Ramnath)
* These authors contributed equally to this work.
KnobGen, a dual-pathway framework that democratizes sketch-based image generation by seamlessly adapting to varying levels of sketch complexity and user skill. KnobGen employs a Coarse-Grained Controller (CGC) module for leveraging high-level semantics from both textual and sketch inputs in the early stages of generation, and a Fine-Grained Controller (FGC) module for detailed refinement later in the process.
- [2024-09-27] 🔥 Initial release of KnobGen code!
To set up the environment and start using KnobGen, please follow these steps:
conda env create -f environment.ymlconda activate knobgen
Our method democratizes sketch-based image generation by effectively handling a broad spectrum of sketch complexity and user drawing ability—from novice sketches to those made by seasoned artists—while maintaining the natural appearance of the image.
