SmolLM3: 3B Multilingual Long-Context Reasoner Released
AI Impact Summary
SmolLM3 is a new 3B open-source language model offering competitive performance against larger 4B models like Qwen3 and Gemma3. This model’s key differentiators are its multilingual capabilities (supporting 6 languages), long context length (up to 128k tokens), and a detailed engineering blueprint released alongside it. The model’s architecture incorporates innovations like grouped query attention (GQA), NoPE, and intra-document masking, all validated through extensive ablation studies, offering a transparent path for developers to understand and replicate its performance.
Affected Systems
Business Impact
Organizations can now leverage a high-performing, open-source language model for applications requiring multilingual support, long context understanding, and advanced reasoning capabilities, potentially reducing reliance on proprietary models.
- Date
- Date not specified
- Change type
- capability
- Severity
- info