Task-Oriented Query Reformulation with Reinforcement Learning

TL;DR

This paper presents a reinforcement learning-based neural query reformulation system that improves search recall for complex queries, demonstrating significant gains over baselines and highlighting potential for further enhancements.

Contribution

Introduces a novel neural network approach trained with reinforcement learning for query reformulation to maximize document recall.

Findings

Achieved 5-20% relative improvement in recall over strong baselines.

Proposed a method to estimate upper-bound performance of query reformulation models.

Identified substantial room for future improvements in query reformulation techniques.

Abstract

Search engines play an important role in our everyday lives by assisting us in finding the information we need. When we input a complex query, however, results are often far from satisfactory. In this work, we introduce a query reformulation system based on a neural network that rewrites a query to maximize the number of relevant documents returned. We train this neural network with reinforcement learning. The actions correspond to selecting terms to build a reformulated query, and the reward is the document recall. We evaluate our approach on three datasets against strong baselines and show a relative improvement of 5-20% in terms of recall. Furthermore, we present a simple method to estimate a conservative upper-bound performance of a model in a particular environment and verify that there is still large room for improvements.