Robust Image Retrieval-based Visual Localization using Kapture

TL;DR

This paper introduces kapture, a flexible data format and toolbox for visual localization, enabling evaluation across multiple datasets and demonstrating high performance with various configurations.

Contribution

We present kapture, a unified data format and toolbox that simplifies dataset integration and evaluation for visual localization and structure-from-motion tasks.

Findings

Our pipeline achieves top rankings on eight public datasets.

Kapture enables versatile use of features, 3D data, and sensor inputs.

Open-source release facilitates future research and benchmarking.

Abstract

Visual localization tackles the challenge of estimating the camera pose from images by using correspondence analysis between query images and a map. This task is computation and data intensive which poses challenges on thorough evaluation of methods on various datasets. However, in order to further advance in the field, we claim that robust visual localization algorithms should be evaluated on multiple datasets covering a broad domain variety. To facilitate this, we introduce kapture, a new, flexible, unified data format and toolbox for visual localization and structure-from-motion (SFM). It enables easy usage of different datasets as well as efficient and reusable data processing. To demonstrate this, we present a versatile pipeline for visual localization that facilitates the use of different local and global features, 3D data (e.g. depth maps), non-vision sensor data (e.g. IMU, GPS, WiFi), and various processing algorithms. Using multiple configurations of the pipeline, we show the great versatility of kapture in our experiments. Furthermore, we evaluate our methods on eight public datasets where they rank top on all and first on many of them. To foster future research, we release code, models, and all datasets used in this paper in the kapture format open source under a permissive BSD license. github.com/naver/kapture, github.com/naver/kapture-localization