A Brief Comparison of Training-Free Multi-Vector Sequence Compression Methods

TL;DR

This paper evaluates training-free sequence compression methods for multi-vector retrieval models, finding token merging more effective than token pruning in reducing index size without sacrificing retrieval quality.

Contribution

It introduces an evaluation of training-free compression techniques specifically for the sequence-length dimension in multi-vector retrieval, highlighting the superiority of token merging.

Findings

Token merging outperforms token pruning in size reduction.

Compression maintains retrieval effectiveness.

Training-free methods are effective for sequence dimension compression.

Abstract

While multi-vector retrieval models outperform single-vector models of comparable size in retrieval quality, their practicality is limited by substantially larger index sizes, driven by the additional sequence-length dimension in their document embeddings. Because document embedding size dictates both memory overhead and query latency, compression is essential for deployment. In this work, we present an evaluation of training-free methods targeting the token sequence length, a dimension unique to multi-vector retrieval. Our findings suggest that token merging is strictly superior to token pruning for reducing index size while maintaining retrieval effectiveness.