🧨 Diffusers: ControlNet integration released

AI Impact Summary

Hugging Face has released ControlNet for Diffusers, enabling users to condition Stable Diffusion image generation with spatial context like depth maps, segmentation maps, and scribbles. This implementation, adapted from the original source code, leverages zero convolution layers for efficient training and allows for switching between different ControlNet models without retraining. The pipeline utilizes the StableDiffusionControlNetPipeline and requires both the pre-trained diffusion model weights and the trained ControlNet weights for inference, adding approximately 700 million parameters compared to a standard Stable Diffusion model.