A Self-Supervised, Differentiable Kalman Filter for Uncertainty-Aware Visual-Inertial Odometry

TL;DR

This paper presents a self-supervised, differentiable Kalman filter framework for visual-inertial odometry that improves robustness and accuracy in challenging conditions by combining learning and classical filtering techniques.

Contribution

It introduces a hybrid VIO system leveraging a differentiable Kalman filter with neural network-based measurements, enabling uncertainty handling and online retraining.

Findings

Operates reliably on degraded visual data where classical methods fail.

Recovers metric scene scale using IMU data, unlike other self-supervised methods.

Outperforms supervised approaches in robustness and accuracy.

Abstract

Visual-inertial odometry (VIO) systems traditionally rely on filtering or optimization-based techniques for egomotion estimation. While these methods are accurate under nominal conditions, they are prone to failure during severe illumination changes, rapid camera motions, or on low-texture image sequences. Learning-based systems have the potential to outperform classical implementations in challenging environments, but, currently, do not perform as well as classical methods in nominal settings. Herein, we introduce a framework for training a hybrid VIO system that leverages the advantages of learning and standard filtering-based state estimation. Our approach is built upon a differentiable Kalman filter, with an IMU-driven process model and a robust, neural network-derived relative pose measurement model. The use of the Kalman filter framework enables the principled treatment of uncertainty at training time and at test time. We show that our self-supervised loss formulation outperforms a similar, supervised method, while also enabling online retraining. We evaluate our system on a visually degraded version of the EuRoC dataset and find that our estimator operates without a significant reduction in accuracy in cases where classical estimators consistently diverge. Finally, by properly utilizing the metric information contained in the IMU measurements, our system is able to recover metric scene scale, while other self-supervised monocular VIO approaches cannot.