Balancing the Budget: Feature Selection and Tracking for Multi-Camera Visual-Inertial Odometry

TL;DR

This paper introduces a multi-camera visual-inertial odometry system that improves motion tracking accuracy in challenging environments by tracking features across cameras and maintaining a fixed feature budget, significantly reducing drift.

Contribution

The paper presents two novel methods for multi-camera feature tracking and feature selection, enhancing accuracy and efficiency in visual-inertial odometry systems.

Findings

Reduces drift rate by up to 80% in translation

Decreases relative pose error by up to 39% in rotation

Effective in challenging environments like underground mines and narrow corridors

Abstract

We present a multi-camera visual-inertial odometry system based on factor graph optimization which estimates motion by using all cameras simultaneously while retaining a fixed overall feature budget. We focus on motion tracking in challenging environments, such as narrow corridors, dark spaces with aggressive motions, and abrupt lighting changes. These scenarios cause traditional monocular or stereo odometry to fail. While tracking motion with extra cameras should theoretically prevent failures, it leads to additional complexity and computational burden. To overcome these challenges, we introduce two novel methods to improve multi-camera feature tracking. First, instead of tracking features separately in each camera, we track features continuously as they move from one camera to another. This increases accuracy and achieves a more compact factor graph representation. Second, we select a fixed budget of tracked features across the cameras to reduce back-end optimization time. We have found that using a smaller set of informative features can maintain the same tracking accuracy. Our proposed method was extensively tested using a hardware-synchronized device consisting of an IMU and four cameras (a front stereo pair and two lateral) in scenarios including: an underground mine, large open spaces, and building interiors with narrow stairs and corridors. Compared to stereo-only state-of-the-art visual-inertial odometry methods, our approach reduces the drift rate, relative pose error, by up to 80% in translation and 39% in rotation.