Learning Interpretable BEV Based VIO without Deep Neural Networks

TL;DR

This paper introduces a fully differentiable, interpretable bird-eye-view based visual-inertial odometry model that does not rely on deep neural networks, enabling end-to-end training and better generalization.

Contribution

The authors propose a novel BEV-based VIO model using differentiable Kalman filtering and camera projection, eliminating the need for deep neural networks and manual parameter tuning.

Findings

Achieves competitive performance with state-of-the-art methods.

Generalizes well to unseen scenes.

Can be trained end-to-end without deep neural networks.

Abstract

Monocular visual-inertial odometry (VIO) is a critical problem in robotics and autonomous driving. Traditional methods solve this problem based on filtering or optimization. While being fully interpretable, they rely on manual interference and empirical parameter tuning. On the other hand, learning-based approaches allow for end-to-end training but require a large number of training data to learn millions of parameters. However, the non-interpretable and heavy models hinder the generalization ability. In this paper, we propose a fully differentiable, and interpretable, bird-eye-view (BEV) based VIO model for robots with local planar motion that can be trained without deep neural networks. Specifically, we first adopt Unscented Kalman Filter as a differentiable layer to predict the pitch and roll, where the covariance matrices of noise are learned to filter out the noise of the IMU raw data. Second, the refined pitch and roll are adopted to retrieve a gravity-aligned BEV image of each frame using differentiable camera projection. Finally, a differentiable pose estimator is utilized to estimate the remaining 3 DoF poses between the BEV frames: leading to a 5 DoF pose estimation. Our method allows for learning the covariance matrices end-to-end supervised by the pose estimation loss, demonstrating superior performance to empirical baselines. Experimental results on synthetic and real-world datasets demonstrate that our simple approach is competitive with state-of-the-art methods and generalizes well on unseen scenes.