MAC-VO: Metrics-aware Covariance for Learning-based Stereo Visual Odometry

TL;DR

MAC-VO introduces a metrics-aware covariance model for stereo visual odometry that improves robustness and accuracy by filtering low-quality features and weighting residuals based on learned uncertainty, outperforming existing methods in challenging environments.

Contribution

The paper presents a novel covariance modeling approach that incorporates learned metrics-aware uncertainty into pose graph optimization for stereo VO, enhancing robustness in difficult conditions.

Findings

Outperforms existing VO algorithms on public benchmarks.

Provides reliable covariance maps indicating pose estimation confidence.

Enhances robustness in environments with varying illumination and feature density.

Abstract

We propose the MAC-VO, a novel learning-based stereo VO that leverages the learned metrics-aware matching uncertainty for dual purposes: selecting keypoint and weighing the residual in pose graph optimization. Compared to traditional geometric methods prioritizing texture-affluent features like edges, our keypoint selector employs the learned uncertainty to filter out the low-quality features based on global inconsistency. In contrast to the learning-based algorithms that model the scale-agnostic diagonal weight matrix for covariance, we design a metrics-aware covariance model to capture the spatial error during keypoint registration and the correlations between different axes. Integrating this covariance model into pose graph optimization enhances the robustness and reliability of pose estimation, particularly in challenging environments with varying illumination, feature density, and motion patterns. On public benchmark datasets, MAC-VO outperforms existing VO algorithms and even some SLAM algorithms in challenging environments. The covariance map also provides valuable information about the reliability of the estimated poses, which can benefit decision-making for autonomous systems.