Modelling Stopping Criteria for Search Results using Poisson Processes

TL;DR

This paper introduces a novel Poisson process-based method for determining stopping criteria in text retrieval, enabling efficient document evaluation by predicting when a desired recall level is likely achieved.

Contribution

The paper proposes a new Poisson process model for stopping criteria that allows users to specify recall and confidence levels, improving over previous techniques.

Findings

Effective in predicting when to stop document evaluation

Outperforms previous methods on a public dataset

Provides customizable recall and probability thresholds

Abstract

Text retrieval systems often return large sets of documents, particularly when applied to large collections. Stopping criteria can reduce the number of these documents that need to be manually evaluated for relevance by predicting when a suitable level of recall has been achieved. In this work, a novel method for determining a stopping criterion is proposed that models the rate at which relevant documents occur using a Poisson process. This method allows a user to specify both a minimum desired level of recall to achieve and a desired probability of having achieved it. We evaluate our method on a public dataset and compare it with previous techniques for determining stopping criteria.