Multi-cam Multi-map Visual Inertial Localization: System, Validation and Dataset

TL;DR

This paper introduces a real-time, causal visual-inertial localization system using multiple cameras and maps, validated on a new challenging dataset, outperforming existing methods in accuracy.

Contribution

The paper presents a novel multi-camera multi-map visual-inertial localization system that ensures real-time, causal pose estimation with bounded error, along with new evaluation metrics and a challenging dataset.

Findings

System achieves higher real-time localization accuracy.

Validated on public benchmarks and a new campus dataset.

Provides open-source code and dataset for community use.

Abstract

Robot control loops require causal pose estimates that depend only on past and present measurements. At each timestep, controllers compute commands using the current pose without waiting for future refinements. While traditional visual SLAM systems achieve high accuracy through retrospective loop closures, these corrections arrive after control decisions were already executed, violating causality. Visual-inertial odometry maintains causality but accumulates unbounded drift over time. To address the distinct requirements of robot control, we propose a multi-camera multi-map visual-inertial localization system providing real-time, causal pose estimation with bounded localization error through continuous map constraints. Since standard trajectory metrics evaluate post-processed trajectories, we analyze the error composition of map-based localization systems and propose a set of evaluation metrics suitable for measuring causal localization performance. To validate our system, we design a multi-camera IMU hardware setup and collect a challenging long-term campus dataset featuring diverse illumination and seasonal conditions. Experimental results on public benchmarks and on our own collected dataset demonstrate that our system provides significantly higher real-time localization accuracy compared to other methods. To benefit the community, we have made both the system and the dataset open source at https://anonymous.4open.science/r/Multi-cam-Multi-map-VILO-7993.