DINO-Mix: Enhancing Visual Place Recognition with Foundational Vision Model and Feature Mixing

TL;DR

This paper introduces DINO-Mix, a novel visual place recognition architecture that combines foundational vision models with feature mixing, significantly improving accuracy in complex environments with lighting, seasonal, and occlusion challenges.

Contribution

The paper proposes DINO-Mix, a new VPR method that leverages foundational vision models and feature aggregation to enhance robustness and accuracy in challenging conditions.

Findings

DINO-Mix achieves top-1 accuracy of 91.75% on Tokyo24/7.

It outperforms state-of-the-art methods with an average accuracy gain of 5.14%.

Demonstrates robustness across lighting, seasonal, and occlusion variations.

Abstract

Utilizing visual place recognition (VPR) technology to ascertain the geographical location of publicly available images is a pressing issue for real-world VPR applications. Although most current VPR methods achieve favorable results under ideal conditions, their performance in complex environments, characterized by lighting variations, seasonal changes, and occlusions caused by moving objects, is generally unsatisfactory. In this study, we utilize the DINOv2 model as the backbone network for trimming and fine-tuning to extract robust image features. We propose a novel VPR architecture called DINO-Mix, which combines a foundational vision model with feature aggregation. This architecture relies on the powerful image feature extraction capabilities of foundational vision models. We employ an MLP-Mixer-based mix module to aggregate image features, resulting in globally robust and generalizable descriptors that enable high-precision VPR. We experimentally demonstrate that the proposed DINO-Mix architecture significantly outperforms current state-of-the-art (SOTA) methods. In test sets having lighting variations, seasonal changes, and occlusions (Tokyo24/7, Nordland, SF-XL-Testv1), our proposed DINO-Mix architecture achieved Top-1 accuracy rates of 91.75%, 80.18%, and 82%, respectively. Compared with SOTA methods, our architecture exhibited an average accuracy improvement of 5.14%.