Efficient large-scale image retrieval with deep feature orthogonality and Hybrid-Swin-Transformers

TL;DR

This paper introduces a new end-to-end pipeline for large-scale landmark image retrieval, combining deep orthogonal feature fusion and hybrid transformer architectures, achieving state-of-the-art results in a major competition.

Contribution

It proposes two novel architectures, DOLG with deep orthogonal fusion and a Hybrid-Swin-Transformer, along with an efficient training method and a discriminative re-ranking technique.

Findings

Won the Google Landmark Competition 2021

Demonstrated superior accuracy in large-scale landmark retrieval

Introduced effective training strategies for complex models

Abstract

We present an efficient end-to-end pipeline for largescale landmark recognition and retrieval. We show how to combine and enhance concepts from recent research in image retrieval and introduce two architectures especially suited for large-scale landmark identification. A model with deep orthogonal fusion of local and global features (DOLG) using an EfficientNet backbone as well as a novel Hybrid-Swin-Transformer is discussed and details how to train both architectures efficiently using a step-wise approach and a sub-center arcface loss with dynamic margins are provided. Furthermore, we elaborate a novel discriminative re-ranking methodology for image retrieval. The superiority of our approach was demonstrated by winning the recognition and retrieval track of the Google Landmark Competition 2021.