Two-Step SPLADE: Simple, Efficient and Effective Approximation of SPLADE

TL;DR

This paper introduces a two-step cascade query processing strategy for SPLADE that significantly improves efficiency, reducing response times up to 40 times without sacrificing effectiveness across diverse datasets.

Contribution

It proposes a novel two-step cascade approach for SPLADE that enhances efficiency while maintaining effectiveness, outperforming previous methods on multiple datasets.

Findings

Up to 30x faster response times on in-domain datasets.

Up to 25x faster response times on out-of-domain datasets.

No significant accuracy loss in 60% of datasets.

Abstract

Learned sparse models such as SPLADE have successfully shown how to incorporate the benefits of state-of-the-art neural information retrieval models into the classical inverted index data structure. Despite their improvements in effectiveness, learned sparse models are not as efficient as classical sparse model such as BM25. The problem has been investigated and addressed by recently developed strategies, such as guided traversal query processing and static pruning, with different degrees of success on in-domain and out-of-domain datasets. In this work, we propose a new query processing strategy for SPLADE based on a two-step cascade. The first step uses a pruned and reweighted version of the SPLADE sparse vectors, and the second step uses the original SPLADE vectors to re-score a sample of documents retrieved in the first stage. Our extensive experiments, performed on 30 different in-domain and out-of-domain datasets, show that our proposed strategy is able to improve mean and tail response times over the original single-stage SPLADE processing by up to $30\times$ and $40\times$, respectively, for in-domain datasets, and by 12x to 25x, for mean response on out-of-domain datasets, while not incurring in statistical significant difference in 60\% of datasets.