NeurIPS 2025 E2LM Competition: Early Training Evaluation of Language Models

AI Impact Summary

NeurIPS is launching a competition focused on evaluating early-stage language model training, specifically targeting scientific knowledge. This competition leverages the Hugging Face ecosystem, utilizing lm-evaluation-harness and Google Colab GPUs, to assess model performance based on signal quality, ranking consistency, and scientific knowledge compliance. The unique evaluation setup, including hidden checkpoints and automated scoring, aims to prevent overly tailored solutions and drive the development of new benchmarks for capturing meaningful signals during LLM early training.