Tightly-Coupled Radar-Visual-Inertial Odometry

TL;DR

This paper presents a tightly coupled radar-visual-inertial odometry system that combines radar, visual, and inertial data within an IEKF to achieve robust state estimation across diverse challenging environments.

Contribution

It introduces a novel fusion approach that integrates radar Doppler, visual features, and IMU data in real-time for improved robustness and accuracy.

Findings

Demonstrates robustness in dark, foggy, and fast-flight conditions

Achieves reliable state estimation in indoor and outdoor experiments

Outperforms traditional VIO in adverse environments

Abstract

Visual-Inertial Odometry (VIO) is a staple for reliable state estimation on constrained and lightweight platforms due to its versatility and demonstrated performance. However, pertinent challenges regarding robust operation in dark, low-texture, obscured environments complicate the use of such methods. Alternatively, Frequency Modulated Continuous Wave (FMCW) radars, and by extension Radar-Inertial Odometry (RIO), offer robustness to these visual challenges, albeit at the cost of reduced information density and worse long-term accuracy. To address these limitations, this work combines the two in a tightly coupled manner, enabling the resulting method to operate robustly regardless of environmental conditions or trajectory dynamics. The proposed method fuses image features, radar Doppler measurements, and Inertial Measurement Unit (IMU) measurements within an Iterated Extended Kalman Filter (IEKF) in real-time, with radar range data augmenting the visual feature depth initialization. The method is evaluated through flight experiments conducted in both indoor and outdoor environments, as well as through challenges to both exteroceptive modalities (such as darkness, fog, or fast flight), thoroughly demonstrating its robustness. The implementation of the proposed method is available at: https://github.com/ntnu-arl/radvio.