Google releases Gemma 2 - new open LLM (9B & 27B)

AI Impact Summary

Google released Gemma 2, a new open large language model (LLM) available in 9B and 27B parameter sizes. These models are based on Google DeepMind’s Gemini and trained on a substantial dataset of 13 trillion tokens for the 27B version and 8 trillion tokens for the 9B version, primarily in English, code, and math. The key advancements include sliding window attention, soft-capping, knowledge distillation, and a novel model merging technique called Warp, offering potential improvements in generation quality and efficiency compared to the original Gemma.