WARP: An Efficient Engine for Multi-Vector Retrieval

TL;DR

WARP is a highly efficient retrieval engine that significantly reduces latency in multi-vector retrieval systems like XTR and ColBERTv2 by employing innovative techniques without sacrificing accuracy.

Contribution

WARP introduces three novel techniques for efficient multi-vector retrieval, enabling faster performance while maintaining high retrieval quality.

Findings

41x reduction in end-to-end latency compared to XTR

3x speedup over ColBERTv2/PLAID

Maintains retrieval quality despite efficiency improvements

Abstract

Multi-vector retrieval methods such as ColBERT and its recent variant, the ConteXtualized Token Retriever (XTR), offer high accuracy but face efficiency challenges at scale. To address this, we present WARP, a retrieval engine that substantially improves the efficiency of retrievers trained with the XTR objective through three key innovations: (1) WARP$_\text{SELECT}$ for dynamic similarity imputation; (2) implicit decompression, avoiding costly vector reconstruction during retrieval; and (3) a two-stage reduction process for efficient score aggregation. Combined with highly-optimized C++ kernels, our system reduces end-to-end latency compared to XTR's reference implementation by 41x, and achieves a 3x speedup over the ColBERTv2/PLAID engine, while preserving retrieval quality.