Attention-Based Query Expansion Learning

TL;DR

This paper introduces an attention-based learning framework for query expansion in image search, which learns how to effectively aggregate images to improve retrieval accuracy across various scenarios.

Contribution

It proposes a discriminative, trainable model using self-attention for query expansion, outperforming existing methods and demonstrating robustness across different regimes.

Findings

Achieves higher accuracy than existing methods on benchmarks.

Consistently performs well under various regimes.

Overcomes limitations of ad-hoc query expansion techniques.

Abstract

Query expansion is a technique widely used in image search consisting in combining highly ranked images from an original query into an expanded query that is then reissued, generally leading to increased recall and precision. An important aspect of query expansion is choosing an appropriate way to combine the images into a new query. Interestingly, despite the undeniable empirical success of query expansion, ad-hoc methods with different caveats have dominated the landscape, and not a lot of research has been done on learning how to do query expansion. In this paper we propose a more principled framework to query expansion, where one trains, in a discriminative manner, a model that learns how images should be aggregated to form the expanded query. Within this framework, we propose a model that leverages a self-attention mechanism to effectively learn how to transfer information between the different images before aggregating them. Our approach obtains higher accuracy than existing approaches on standard benchmarks. More importantly, our approach is the only one that consistently shows high accuracy under different regimes, overcoming caveats of existing methods.