Weaviate v1.18: Introducing HNSW+PQ Vector Compression
AI Impact Summary
Weaviate is introducing HNSW+PQ vector compression, leveraging Product Quantization (PQ) to reduce memory footprint and cost. This involves moving vector and graph representations to disk, fetching compressed vectors from disk during searches and only loading full vectors when needed. This approach addresses the scalability challenges of storing large vector datasets, particularly when dealing with datasets like Gist or DeepImage96, by significantly reducing memory requirements and minimizing disk read latency.
Affected Systems
Business Impact
Organizations using Weaviate can significantly reduce infrastructure costs and improve performance for large vector datasets by utilizing HNSW+PQ compression.
- Date
- Date not specified
- Change type
- capability
- Severity
- info