Counterfactual Learning To Rank for Utility-Maximizing Query Autocompletion

TL;DR

This paper introduces a counterfactual learning to rank method for query autocompletion that optimizes for downstream retrieval performance rather than mimicking user behavior, supported by theoretical guarantees and empirical validation.

Contribution

It proposes a novel counterfactual learning approach that directly optimizes query suggestions for retrieval quality, overcoming limitations of traditional behavior-mimicking methods.

Findings

The method achieves improved retrieval performance in experiments.

The approach provides unbiased estimates of query suggestion quality.

Theoretical guarantees support the method's validity.

Abstract

Conventional methods for query autocompletion aim to predict which completed query a user will select from a list. A shortcoming of this approach is that users often do not know which query will provide the best retrieval performance on the current information retrieval system, meaning that any query autocompletion methods trained to mimic user behavior can lead to suboptimal query suggestions. To overcome this limitation, we propose a new approach that explicitly optimizes the query suggestions for downstream retrieval performance. We formulate this as a problem of ranking a set of rankings, where each query suggestion is represented by the downstream item ranking it produces. We then present a learning method that ranks query suggestions by the quality of their item rankings. The algorithm is based on a counterfactual learning approach that is able to leverage feedback on the items (e.g., clicks, purchases) to evaluate query suggestions through an unbiased estimator, thus avoiding the assumption that users write or select optimal queries. We establish theoretical support for the proposed approach and provide learning-theoretic guarantees. We also present empirical results on publicly available datasets, and demonstrate real-world applicability using data from an online shopping store.