360DVO: Deep Visual Odometry for Monocular 360-Degree Camera

TL;DR

The paper introduces 360DVO, a deep learning-based monocular 360-degree visual odometry system that uses a distortion-aware feature extractor and differentiable bundle adjustment to improve robustness and accuracy in challenging scenarios.

Contribution

It presents the first deep learning framework for omnidirectional visual odometry, including a novel distortion-aware feature extractor and a differentiable bundle adjustment module.

Findings

Outperforms state-of-the-art methods in robustness and accuracy

Demonstrates 50% improvement in robustness

Achieves 37.5% higher accuracy

Abstract

Monocular omnidirectional visual odometry (OVO) systems leverage 360-degree cameras to overcome field-of-view limitations of perspective VO systems. However, existing methods, reliant on handcrafted features or photometric objectives, often lack robustness in challenging scenarios, such as aggressive motion and varying illumination. To address this, we present 360DVO, the first deep learning-based OVO framework. Our approach introduces a distortion-aware spherical feature extractor (DAS-Feat) that adaptively learns distortion-resistant features from 360-degree images. These sparse feature patches are then used to establish constraints for effective pose estimation within a novel omnidirectional differentiable bundle adjustment (ODBA) module. To facilitate evaluation in realistic settings, we also contribute a new real-world OVO benchmark. Extensive experiments on this benchmark and public synthetic datasets (TartanAir V2 and 360VO) demonstrate that 360DVO surpasses state-of-the-art baselines (including 360VO and OpenVSLAM), improving robustness by 50% and accuracy by 37.5%. Homepage: https://chris1004336379.github.io/360DVO-homepage