Hugging Face Hub adds Xet storage to replace LFS for large files

AI Impact Summary

Hugging Face is deploying Xet storage on the Hub to replace LFS for large repository data, using content-defined chunking to deduplicate at ~64KB granularity. The migration reduced dependence on full-file reuploads, demonstrated real-world throughput gains (e.g., sub-second dashboard data transfers versus minutes for large blocks), and required coordinated changes across the CAS, 64KB chunking, and LFS Bridge components. This is a significant capability shift for large-file workflows, but teams will need to adapt their tooling to use Xet directly or via the LFS Bridge for backward compatibility.