On the Equilibrium of Query Reformulation and Document Retrieval

TL;DR

This paper models the interaction between query reformulation and document retrieval as a strategic game, providing a theoretical framework and practical algorithms that improve retrieval performance.

Contribution

It introduces a game-theoretic equilibrium model for IR, analyzing different feedback algorithms and proposing new algorithms to reach optimal retrieval states.

Findings

Equilibrium theory explains IR interactions and objectives.

Partnership game occurs with standard relevance feedback.

General-sum game arises with pseudo relevance feedback.

Abstract

In this paper, we study jointly query reformulation and document relevance estimation, the two essential aspects of information retrieval (IR). Their interactions are modelled as a two-player strategic game: one player, a query formulator, taking actions to produce the optimal query, is expected to maximize its own utility with respect to the relevance estimation of documents produced by the other player, a retrieval modeler; simultaneously, the retrieval modeler, taking actions to produce the document relevance scores, needs to optimize its likelihood from the training data with respect to the refined query produced by the query formulator. Their equilibrium or equilibria will be reached when both are the best responses to each other. We derive our equilibrium theory of IR using normal-form representations: when a standard relevance feedback algorithm is coupled with a retrieval model, they would share the same objective function and thus form a partnership game; by contrast, pseudo relevance feedback pursues a rather different objective than that of retrieval models, therefore the interaction between them would lead to a general-sum game (though implicitly collaborative). Our game-theoretical analyses not only yield useful insights into the two major aspects of IR, but also offer new practical algorithms for achieving the equilibrium state of retrieval which have been shown to bring consistent performance improvements in both text retrieval and item recommendation.