Weaviate AI 数据库：如何仅花费一半预算来维持向量索引运行？
TL;DR · AI 摘要
Weaviate提出了一种新的内存管理方案HFresh，通过将向量存储在磁盘上并仅保留紧凑的中心索引来降低内存使用。
核心要点
- HFresh节省内存达50%以上
- HFresh适用于大规模数据集
- HFresh成本敏感部署更灵活
结构提纲
按章节快速跳转。
思维导图
用一张图看清主题之间的关系。
查看大纲文本（无障碍 / 无 JS 友好）
- HFresh内存管理方案
- 应用场景
- 性能优势
- 实施方式
金句 / Highlights
值得收藏与分享的关键句。
HFresh通过将向量存储在磁盘上并仅保留紧凑的中心索引来降低内存使用，显著节省50%以上。
HFresh适用于大规模数据集和成本敏感部署，具有显著的成本效益。
HFresh通过将向量划分为小区域并使用紧凑的中心索引来实现高效搜索，具有可预测的延迟。
There's a better way.
HNSW is the gold standard for vector search, but it needs everything in memory. As datasets grow, that gets expensive.
HFresh flips the model by storing vectors on disk while https://t.co/VSGzgQOEW7" / X
Spending half your budget on memory just to keep your vector index running? There's a better way. HNSW is the gold standard for vector search, but it needs everything in memory. As datasets grow, that gets expensive. HFresh flips the model by storing vectors on disk while keeping only a compact centroid index in memory. 𝗛𝗼𝘄 𝗶𝘁 𝘄𝗼𝗿𝗸𝘀 • Divides vectors into small regions called postings • Uses a compact in-memory HNSW index over centroids to identify relevant regions • Fetches only the relevant postings from disk for search • Applies Rotational Quantization at two levels for compression The result is significantly lower memory usage with predictable latency, even at billion-vector scale 𝗪𝗵𝗲𝗻 𝘁𝗼 𝗰𝗼𝗻𝘀𝗶𝗱𝗲𝗿 𝗛𝗙𝗿𝗲𝘀𝗵 𝗟𝗮𝗿𝗴𝗲 𝗱𝗮𝘁𝗮𝘀𝗲𝘁𝘀 𝘄𝗶𝘁𝗵 𝗵𝗶𝗴𝗵-𝗱𝗶𝗺𝗲𝗻𝘀𝗶𝗼𝗻𝗮𝗹 𝗲𝗺𝗯𝗲𝗱𝗱𝗶𝗻𝗴𝘀 — Memory savings become substantial compared to HNSW 𝗖𝗼𝘀𝘁-𝘀𝗲𝗻𝘀𝗶𝘁𝗶𝘃𝗲 𝗱𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁𝘀 — Run the same workload on smaller infrastructure 𝗪𝗿𝗶𝘁𝗲-𝗵𝗲𝗮𝘃𝘆 𝘄𝗼𝗿𝗸𝗹𝗼𝗮𝗱𝘀 — The cluster-based design avoids the write amplification that HNSW can experience during large imports HFresh is currently a technical preview, so we recommend testing in non-production environments first. Read our blog for more details: weaviate.io/blog/weaviate- Try HFresh now in Weaviate Cloud console.weaviate.cloud/signin?utm_sou
