StarCoder2-Instruct: Fully Transparent Self-Alignment for Code Generation

AI Impact Summary

StarCoder2-Instruct represents a significant advancement in code generation by employing a fully self-aligned training pipeline, eliminating reliance on potentially restricted or expensive teacher models like GPT-4. This approach, utilizing a permissive and transparent data generation process, results in a 72.6 HumanEval score, surpassing CodeLlama-70B-Instruct and demonstrating the effectiveness of self-alignment for code LLMs. The open-source nature of this model and its training data offers a valuable resource for the broader AI community.