Benchmarking Visual Localization for Autonomous Navigation

TL;DR

This paper presents a simulator-based benchmark for visual localization in autonomous navigation, examining how environmental variables impact the performance of state-of-the-art methods within a complete navigation stack.

Contribution

It introduces the first comprehensive benchmark evaluating modern visual localization methods in a full autonomous navigation system under varying conditions.

Findings

Major variation in method suitability depending on environmental factors

Benchmark enables systematic evaluation of localization methods in navigation tasks

Provides a publicly available tool for future research

Abstract

This work introduces a simulator-based benchmark for visual localization in the autonomous navigation context. The dynamic benchmark enables investigation of how variables such as the time of day, weather, and camera perspective affect the navigation performance of autonomous agents that utilize visual localization for closed-loop control. The experimental part of the paper studies the effects of four such variables by evaluating state-of-the-art visual localization methods as part of the motion planning module of an autonomous navigation stack. The results show major variation in the suitability of the different methods for vision-based navigation. To the authors' best knowledge, the proposed benchmark is the first to study modern visual localization methods as part of a complete navigation stack. We make the benchmark available at https://github.com/lasuomela/carla_vloc_benchmark.