An Alternative to FLOPS Regularization to Effectively Productionize SPLADE-Doc

TL;DR

This paper introduces DF-FLOPS, a new regularization method for Sparse Retrieval models that reduces high-frequency term usage, significantly lowering retrieval latency while maintaining retrieval effectiveness in production environments.

Contribution

The paper proposes DF-FLOPS, an alternative to FLOPS regularization, which penalizes high document frequency terms to improve retrieval speed and practicality in production search engines.

Findings

DF-FLOPS reduces high-DF terms and retrieval latency by around 10x.

Maintains effectiveness with only a 2.2-point drop in MRR@10.

Improves performance in cross-domain retrieval tasks.

Abstract

Learned Sparse Retrieval (LSR) models encode text as weighted term vectors, which need to be sparse to leverage inverted index structures during retrieval. SPLADE, the most popular LSR model, uses FLOPS regularization to encourage vector sparsity during training. However, FLOPS regularization does not ensure sparsity among terms - only within a given query or document. Terms with very high Document Frequencies (DFs) substantially increase latency in production retrieval engines, such as Apache Solr, due to their lengthy posting lists. To address the issue of high DFs, we present a new variant of FLOPS regularization: DF-FLOPS. This new regularization technique penalizes the usage of high-DF terms, thereby shortening posting lists and reducing retrieval latency. Unlike other inference-time sparsification methods, such as stopword removal, DF-FLOPS regularization allows for the selective inclusion of high-frequency terms in cases where the terms are truly salient. We find that DF-FLOPS successfully reduces the prevalence of high-DF terms and lowers retrieval latency (around 10x faster) in a production-grade engine while maintaining effectiveness both in-domain (only a 2.2-point drop in MRR@10) and cross-domain (improved performance in 12 out of 13 tasks on which we tested). With retrieval latencies on par with BM25, this work provides an important step towards making LSR practical for deployment in production-grade search engines.