Densifying Sparse Representations for Passage Retrieval by Representational Slicing

TL;DR

This paper introduces a training-free method to densify sparse text representations, making them more effective for retrieval and easily combinable with dense representations, improving retrieval performance.

Contribution

It proposes a novel densification technique for sparse representations that is interpretable, training-free, and compatible with dense models for improved retrieval.

Findings

Densified sparse representations improve retrieval effectiveness.

Combining DSRs with dense models balances efficiency and accuracy.

The approach is interpretable and does not require additional training.

Abstract

Learned sparse and dense representations capture different successful approaches to text retrieval and the fusion of their results has proven to be more effective and robust. Prior work combines dense and sparse retrievers by fusing their model scores. As an alternative, this paper presents a simple approach to densifying sparse representations for text retrieval that does not involve any training. Our densified sparse representations (DSRs) are interpretable and can be easily combined with dense representations for end-to-end retrieval. We demonstrate that our approach can jointly learn sparse and dense representations within a single model and then combine them for dense retrieval. Experimental results suggest that combining our DSRs and dense representations yields a balanced tradeoff between effectiveness and efficiency.