Neural Passage Quality Estimation for Static Pruning

TL;DR

This paper introduces neural methods to predict passage quality independently of queries, enabling significant corpus pruning that reduces resource consumption while maintaining search effectiveness.

Contribution

It presents novel neural techniques for query-agnostic passage quality estimation, allowing effective corpus pruning in neural search engines.

Findings

Prunes over 25% of passages without losing effectiveness

Reduces computational resources, power, and carbon footprint

Enables lightweight pre-pruning before encoding steps

Abstract

Neural networks -- especially those that use large, pre-trained language models -- have improved search engines in various ways. Most prominently, they can estimate the relevance of a passage or document to a user's query. In this work, we depart from this direction by exploring whether neural networks can effectively predict which of a document's passages are unlikely to be relevant to any query submitted to the search engine. We refer to this query-agnostic estimation of passage relevance as a passage's quality. We find that our novel methods for estimating passage quality allow passage corpora to be pruned considerably while maintaining statistically equivalent effectiveness; our best methods can consistently prune >25% of passages in a corpora, across various retrieval pipelines. Such substantial pruning reduces the operating costs of neural search engines in terms of computing resources, power usage, and carbon footprint -- both when processing queries (thanks to a smaller index size) and when indexing (lightweight models can prune low-quality passages prior to the costly dense or learned sparse encoding step). This work sets the stage for developing more advanced neural "learning-what-to-index" methods.