Differential Transformer V2 (DIFF V2) — Faster Inference & Training Stability

AI Impact Summary

Differential Transformer V2 (DIFF V2) represents a significant architectural shift focused on inference efficiency and training stability for large language models. Key improvements include doubling the query heads without increasing key-value heads, leveraging FlashAttention for faster decoding, and removing the per-head RMSNorm to mitigate training instability, particularly at high learning rates. This design allows DIFF V2 to achieve speeds comparable to standard Transformers while reducing the risk of gradient spikes and numerical issues, making it suitable for production-scale LLM training.