CoProU-VO: Combining Projected Uncertainty for End-to-End Unsupervised Monocular Visual Odometry

TL;DR

CoProU-VO introduces a novel end-to-end method that propagates and combines uncertainty across frames to improve unsupervised monocular visual odometry, especially in dynamic scenes, using vision transformers.

Contribution

It proposes a new probabilistic approach to combine target and reference frame uncertainties for better dynamic scene handling in unsupervised VO.

Findings

Significant performance improvements on KITTI and nuScenes datasets.

Effective in challenging highway scenes with dynamic objects.

Ablation studies confirm the importance of cross-frame uncertainty propagation.

Abstract

Visual Odometry (VO) is fundamental to autonomous navigation, robotics, and augmented reality, with unsupervised approaches eliminating the need for expensive ground-truth labels. However, these methods struggle when dynamic objects violate the static scene assumption, leading to erroneous pose estimations. We tackle this problem by uncertainty modeling, which is a commonly used technique that creates robust masks to filter out dynamic objects and occlusions without requiring explicit motion segmentation. Traditional uncertainty modeling considers only single-frame information, overlooking the uncertainties across consecutive frames. Our key insight is that uncertainty must be propagated and combined across temporal frames to effectively identify unreliable regions, particularly in dynamic scenes. To address this challenge, we introduce Combined Projected Uncertainty VO (CoProU-VO), a novel end-to-end approach that combines target frame uncertainty with projected reference frame uncertainty using a principled probabilistic formulation. Built upon vision transformer backbones, our model simultaneously learns depth, uncertainty estimation, and camera poses. Consequently, experiments on the KITTI and nuScenes datasets demonstrate significant improvements over previous unsupervised monocular end-to-end two-frame-based methods and exhibit strong performance in challenging highway scenes where other approaches often fail. Additionally, comprehensive ablation studies validate the effectiveness of cross-frame uncertainty propagation.