Divide&Classify: Fine-Grained Classification for City-Wide Visual Place Recognition

TL;DR

Divide&Classify introduces a classification-based approach for city-wide visual place recognition, achieving fast inference and high accuracy, and can enhance existing retrieval methods by over 20 times in speed.

Contribution

The paper proposes a novel partitioning scheme and ensemble classifiers with prototypes learned via angular margin loss for fine-grained city-wide recognition.

Findings

D&C achieves competitive accuracy with retrieval methods.

Pairing D&C with retrieval pipelines speeds up computation by over 20 times.

D&C improves recall in large-scale visual place recognition.

Abstract

Visual Place recognition is commonly addressed as an image retrieval problem. However, retrieval methods are impractical to scale to large datasets, densely sampled from city-wide maps, since their dimension impact negatively on the inference time. Using approximate nearest neighbour search for retrieval helps to mitigate this issue, at the cost of a performance drop. In this paper we investigate whether we can effectively approach this task as a classification problem, thus bypassing the need for a similarity search. We find that existing classification methods for coarse, planet-wide localization are not suitable for the fine-grained and city-wide setting. This is largely due to how the dataset is split into classes, because these methods are designed to handle a sparse distribution of photos and as such do not consider the visual aliasing problem across neighbouring classes that naturally arises in dense scenarios. Thus, we propose a partitioning scheme that enables a fast and accurate inference, preserving a simple learning procedure, and a novel inference pipeline based on an ensemble of novel classifiers that uses the prototypes learned via an angular margin loss. Our method, Divide&Classify (D&C), enjoys the fast inference of classification solutions and an accuracy competitive with retrieval methods on the fine-grained, city-wide setting. Moreover, we show that D&C can be paired with existing retrieval pipelines to speed up computations by over 20 times while increasing their recall, leading to new state-of-the-art results.