DOLG: Single-Stage Image Retrieval with Deep Orthogonal Fusion of Local and Global Features

TL;DR

This paper introduces DOLG, a novel single-stage image retrieval framework that effectively fuses local and global features into compact representations, achieving state-of-the-art results.

Contribution

The paper proposes a deep orthogonal fusion method that integrates local and global features in a single end-to-end trainable model for image retrieval.

Findings

Achieves state-of-the-art performance on Revisited Oxford and Paris datasets.

Effectively combines local and global features in a single, end-to-end framework.

Outperforms previous two-stage retrieval methods in accuracy.

Abstract

Image Retrieval is a fundamental task of obtaining images similar to the query one from a database. A common image retrieval practice is to firstly retrieve candidate images via similarity search using global image features and then re-rank the candidates by leveraging their local features. Previous learning-based studies mainly focus on either global or local image representation learning to tackle the retrieval task. In this paper, we abandon the two-stage paradigm and seek to design an effective single-stage solution by integrating local and global information inside images into compact image representations. Specifically, we propose a Deep Orthogonal Local and Global (DOLG) information fusion framework for end-to-end image retrieval. It attentively extracts representative local information with multi-atrous convolutions and self-attention at first. Components orthogonal to the global image representation are then extracted from the local information. At last, the orthogonal components are concatenated with the global representation as a complementary, and then aggregation is performed to generate the final representation. The whole framework is end-to-end differentiable and can be trained with image-level labels. Extensive experimental results validate the effectiveness of our solution and show that our model achieves state-of-the-art image retrieval performances on Revisited Oxford and Paris datasets.