Towards Encrypted Large Language Models with FHE — GPT2 Demonstration

AI Impact Summary

OpenAI is exploring the use of Fully Homomorphic Encryption (FHE) to enable large language model (LLM) inference on encrypted data, addressing privacy concerns particularly in sensitive sectors like healthcare and finance. This involves adapting the Hugging Face transformers library, specifically the GPT2 model, to incorporate FHE-friendly operators and quantization techniques. The demo shows that a quantized LLM model implemented with FHE maintains 96% of the original accuracy with 4-bit quantization, highlighting the potential for practical FHE-based LLM deployments, though performance remains a challenge due to the high computational cost of PBS operations.