Open Arabic LLM Leaderboard 2: native Arabic tasks replace translated benchmarks

AI Impact Summary

Open Arabic LLM Leaderboard 2 shifts the benchmarking paradigm from translated benchmarks to native Arabic tasks, emphasizing Arabic morphology, dialect diversity, and real-world usage. The initiative ties together OALL, AraGen, Balsam Index, and SEAL under a more unified, transparent benchmarking platform, with HuggingFace hosting spaces that facilitate reproducible evaluations. A bug in the AlGhafa task previously affected rankings; its remediation alongside updated native tasks will likely trigger ranking movements and reveal gaps in dialectal robustness among current models.