Query expansion with artificially generated texts
Vincent Claveau

TL;DR
This paper demonstrates that using GPT-2 for query expansion significantly improves document retrieval performance, achieving over 10% MAP gains and outperforming traditional expansion methods.
Contribution
The study introduces a novel approach of leveraging GPT-2 for automatic query expansion, showing its effectiveness over existing methods in IR.
Findings
Query expansion with GPT-2 yields +10% MAP improvements.
GPT-2 based expansion outperforms LM+RM3 baseline.
The approach is easy to implement with available GPT tools.
Abstract
A well-known way to improve the performance of document retrieval is to expand the user's query. Several approaches have been proposed in the literature, and some of them are considered as yielding state-of-the-art results in IR. In this paper, we explore the use of text generation to automatically expand the queries. We rely on a well-known neural generative model, GPT-2, that comes with pre-trained models for English but can also be fine-tuned on specific corpora. Through different experiments, we show that text generation is a very effective way to improve the performance of an IR system, with a large margin (+10% MAP gains), and that it outperforms strong baselines also relying on query expansion (LM+RM3). This conceptually simple approach can easily be implemented on any IR system thanks to the availability of GPT code and models.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Information Retrieval and Search Behavior
MethodsLinear Layer · Cosine Annealing · Discriminative Fine-Tuning · Attention Is All You Need · Byte Pair Encoding · Layer Normalization · Dropout · Weight Decay · Dense Connections · Linear Warmup With Cosine Annealing
