Fine-tuning LLMs to 1.58bit: Microsoft's BitNet architecture

AI Impact Summary

OpenAI is introducing a novel LLM quantization technique, BitNet, which represents parameters with ternary values (-1, 0, 1) to achieve extreme compression (1.58 bits per parameter). This architecture, developed by Microsoft Research, utilizes INT8 addition calculations during matrix multiplication, offering a theoretical 71.4x reduction in energy consumption compared to LLaMA. Fine-tuning a Llama3 8B model with this method demonstrates strong performance on MMLU benchmarks, highlighting the potential for efficient LLM deployment.