LoRA Inference Mutualization in Inference API reduces warm-up to 3s for Stable Diffusion LoRAs

AI Impact Summary

The Inference API now mutualizes LoRA serving by keeping a shared Stable Diffusion XL base model warm and dynamically loading/unloading per-LoRA adapters on demand. It leverages Diffusers library capabilities (load_lora_weights, fuse_lora, unload_lora_weights, unfuse_lora) to merge adapters with the base model in memory, enabling hundreds of LoRAs to be served from a small pool of base deployments. This approach reduces warm-up overhead (25s down to 3s) and cuts per-request latency (35s down to 13s), enabling scalable, cost-efficient LoRA serving for thousands of adapters on limited GPU resources.