多向量检索赢在基准测试上，在生产中却常常失败
TL;DR · AI 摘要
多向量检索在基准测试中表现优异，但在生产环境中常因评分机制与系统实现不匹配而效果下降。
核心要点
- 多向量模型在生产中表现差的主要原因是评分机制与检索系统的不匹配。
- Milvus 建议通过调整相似度聚合方式来对齐模型设计与系统行为。
- 实际部署时需验证查询与文档向量的交互逻辑是否符合模型训练假设。
结构提纲
按章节快速跳转。
多向量检索在基准测试中优于稠密检索，但在生产中常表现更差。
多向量模型依赖细粒度相关性评分，但系统常使用简化聚合导致失真。
通过调整向量聚合策略可恢复多向量模型的实际性能优势。
思维导图
用一张图看清主题之间的关系。
查看大纲文本（无障碍 / 无 JS 友好）
- 多向量检索的生产挑战
- 性能悖论
- 基准胜出
- 生产失利
- 根本原因
- 评分机制错配
- 聚合方式不当
- 解决路径
- 对齐模型与系统
- 优化相似度计算
金句 / Highlights
值得收藏与分享的关键句。
多向量检索赢在基准测试上，在生产中却常常失败。
罪魁祸首几乎总是多向量模型对文档打分的方式与系统实现之间的不匹配。
当团队部署多向量模型时，结果有时比普通稠密检索还差。
𝗠𝘂𝗹𝘁𝗶-𝘃𝗲𝗰𝘁𝗼𝗿 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝘄𝗶𝗻𝘀 𝗼𝗻 𝗯𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸𝘀. 𝗜𝗻 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻, 𝗶𝘁 𝗼𝗳𝘁𝗲𝗻 𝗹𝗼𝘀𝗲𝘀 https://t.co/RndmKyCuCy" / X
In this previous post (x.com/milvusio/statu), we talked about why multi-vector models sometimes have worse results than plain dense retrieval. 𝗠𝘂𝗹𝘁𝗶-𝘃𝗲𝗰𝘁𝗼𝗿 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹 𝘄𝗶𝗻𝘀 𝗼𝗻 𝗯𝗲𝗻𝗰𝗵𝗺𝗮𝗿𝗸𝘀. 𝗜𝗻 𝗽𝗿𝗼𝗱𝘂𝗰𝘁𝗶𝗼𝗻, 𝗶𝘁 𝗼𝗳𝘁𝗲𝗻 𝗹𝗼𝘀𝗲𝘀 𝘁𝗼 𝗽𝗹𝗮𝗶𝗻 𝗱𝗲𝗻𝘀𝗲 𝗿𝗲𝘁𝗿𝗶𝗲𝘃𝗮𝗹. 𝗛𝗲𝗿𝗲'𝘀 𝘄𝗵𝘆, 𝗮𝗻𝗱 𝘄𝗵𝗲𝗻 𝘁𝗵𝗲 𝗮𝗱𝘃𝗮𝗻𝘁𝗮𝗴𝗲 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝘀𝘂𝗿𝘃𝗶𝘃𝗲𝘀. 𝗧𝗵𝗲 𝗮𝗻𝘀𝘄𝗲𝗿 𝘀𝘁𝗮𝗿𝘁𝘀 𝘄𝗶𝘁𝗵 𝘄𝗵𝗮𝘁'𝘀 𝗮𝗰𝘁𝘂𝗮𝗹𝗹𝘆 𝗯𝗲𝗶𝗻𝗴 𝗺𝗲𝗮𝘀𝘂𝗿𝗲𝗱: brute-force exact scoring — every query token compared against every document token. Under those conditions, multi-vector almost always beats dense. The gap is modest on simple text datasets, larger on complex ones, and very large on multimodal data. Production can't do exact scoring; corpora are too large. Both dense and multi-vector fall back to approximate search — but the approximation gap isn't symmetric. Dense ANN (HNSW, IVF) is mature enough to closely track brute-force. Multi-vector approximate search isn't: it uses compressed or aggregated representations to generate candidates first, then re-scores only those. The approximation loss is much larger, and documents that would have ranked high under exact MaxSim often don't make the candidate shortlist at all. 𝗧𝗵𝗲 𝗱𝗮𝘁𝗮 𝘀𝗵𝗼𝘄𝘀 𝗲𝘅𝗮𝗰𝘁𝗹𝘆 𝘄𝗵𝗲𝗿𝗲 𝘁𝗵𝗲𝗿𝗲 𝗶𝘀 𝗼𝗿 𝗶𝘀𝗻'𝘁 𝗹𝗼𝘀𝘀. 𝗛𝗼𝘄 𝗺𝘂𝗰𝗵 𝗼𝗳 𝘁𝗵𝗲 𝗲𝘅𝗮𝗰𝘁-𝘀𝗰𝗼𝗿𝗶𝗻𝗴 𝗮𝗱𝘃𝗮𝗻𝘁𝗮𝗴𝗲 𝘀𝘂𝗿𝘃𝗶𝘃𝗲𝘀 𝗱𝗲𝗽𝗲𝗻𝗱𝘀 𝗼𝗻 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁 𝗹𝗲𝗻𝗴𝘁𝗵 𝗮𝗻𝗱 𝗾𝘂𝗲𝗿𝘆 𝗰𝗼𝗺𝗽𝗹𝗲𝘅𝗶𝘁𝘆: 
MS MARCO (avg. 87 tokens/doc) — ~1pt lead under exact scoring. Approximate search erased it entirely. 
SciFact (avg. 360 tokens/doc) — 3–9pt exact lead. Approximate search erased it too. Uniformly long documents give token-level matching less to exploit than you'd expect. 
TREC-COVID (avg. 236 tokens/doc, complex biomedical queries) — 12–17pt exact lead. 7–12pts survived. Long docs plus hard queries gave MaxSim enough signal to hold up. 
LoTTE (heavy length tail, up to 4,000 tokens/doc) — results split sharply by model and strategy. Length alone doesn't explain it. 𝗦𝗼 𝘄𝗵𝗲𝗻 𝘁𝗼 𝘂𝘀𝗲 𝘄𝗵𝗶𝗰𝗵? 𝗧𝗵𝗲 𝗿𝘂𝗹𝗲 𝗼𝗳 𝘁𝗵𝘂𝗺𝗯: → Short docs, simple queries: dense retrieval wins. Skip the complexity. → Long docs, complex queries: multi-vector holds up. The gain is worth it. → Mixed-length corpus: your model and strategy pairing determines the outcome — more on that next.
Quote
@milvusio
May 8
Sometimes, when teams deploy a multi-vector model, their results come back worse than plain dense retrieval. 𝗧𝗵𝗲 𝗰𝘂𝗹𝗽𝗿𝗶𝘁 𝗶𝘀 𝗮𝗹𝗺𝗼𝘀𝘁 𝗮𝗹𝘄𝗮𝘆𝘀 𝗮 𝗺𝗶𝘀𝗺𝗮𝘁𝗰𝗵 𝗯𝗲𝘁𝘄𝗲𝗲𝗻 𝗵𝗼𝘄 𝗺𝘂𝗹𝘁𝗶-𝘃𝗲𝗰𝘁𝗼𝗿 𝗺𝗼𝗱𝗲𝗹𝘀 𝘀𝗰𝗼𝗿𝗲 𝗱𝗼𝗰𝘂𝗺𝗲𝗻𝘁𝘀 𝗮𝗻𝗱
