Open LLM Leaderboard: Community fine-tunes more carbon-efficient than official models at similar performance

AI Impact Summary

Open LLM Leaderboard findings show CO₂ emissions scale with model size and inference time under a uniform 8-GPU setup, but efficiency gains come from community fine-tunes and smaller models. Community fine-tunes tend to match or exceed official fine-tunes on leaderboard scores while using less energy, indicating the value of task-specific adaptations. MoEs generally underperform on score-per-CO₂, and instruction-tuned models can be more verbose, increasing energy use. A Colab notebook is provided to reproduce results and compare base vs fine-tuned variants.