Falcon-H1 Open-Source Hybrid-Head LLM Family Release (0.5B–34B)

AI Impact Summary

Falcon-H1 introduces six open-weight models (0.5B, 1.5B, 1.5B-Deep, 3B, 7B, 34B) with base and instruction-tuned variants, under a hybrid attention + SSM (Mamba-2) design and a configurable attention/SSM ratio to boost inference speed and memory efficiency. The series supports 256K context and 18 languages via a multilingual tokenizer, signaling strong suitability for long-document processing and multilingual applications while aiming for edge-to-large deployments. As an Apache 2.0 open-weight release, Falcon-H1 provides an alternative to traditional transformer baselines, enabling teams to evaluate long-context, cost-performance tradeoffs and plan workloads around the new architecture and data training strategies.