Hugging Face migrates Hub to Xet storage backend

AI Impact Summary

Hugging Face is migrating the Hub’s storage backend from Git LFS to Xet, a significant change impacting how repositories are stored and accessed. This migration, already underway with over 1 million users and 1 million repositories, involves a complex, automated process of migrating data in the background without disrupting existing workflows. The team has proactively addressed scaling challenges encountered during initial migrations, demonstrating a robust and adaptable infrastructure capable of handling massive datasets and high traffic volumes. This migration ensures the Hub can continue to scale with the growing demands of AI builders and provides a more efficient storage solution.