CrowdDriven: A New Challenging Dataset for Outdoor Visual Localization

TL;DR

CrowdDriven introduces a challenging outdoor visual localization dataset sourced from crowds, covering diverse regions and devices, highlighting current methods' limitations and aiding future improvements.

Contribution

It presents a novel, diverse, crowd-sourced benchmark for outdoor visual localization, emphasizing failure cases and providing tools for annotation and reference pose generation.

Findings

State-of-the-art methods fail on the hardest parts of the dataset.

The dataset covers a wide range of geographical regions and devices.

Current algorithms struggle under diverse outdoor conditions.

Abstract

Visual localization is the problem of estimating the position and orientation from which a given image (or a sequence of images) is taken in a known scene. It is an important part of a wide range of computer vision and robotics applications, from self-driving cars to augmented/virtual reality systems. Visual localization techniques should work reliably and robustly under a wide range of conditions, including seasonal, weather, illumination and man-made changes. Recent benchmarking efforts model this by providing images under different conditions, and the community has made rapid progress on these datasets since their inception. However, they are limited to a few geographical regions and often recorded with a single device. We propose a new benchmark for visual localization in outdoor scenes, using crowd-sourced data to cover a wide range of geographical regions and camera devices with a focus on the failure cases of current algorithms. Experiments with state-of-the-art localization approaches show that our dataset is very challenging, with all evaluated methods failing on its hardest parts. As part of the dataset release, we provide the tooling used to generate it, enabling efficient and effective 2D correspondence annotation to obtain reference poses.