Full Surround Monodepth from Multiple Cameras

TL;DR

This paper introduces a self-supervised method for monocular depth and ego-motion estimation using multiple cameras to achieve full 360-degree scene coverage, comparable to LiDAR, with dense and scale-aware point clouds.

Contribution

It extends monocular depth estimation to multi-camera rigs, incorporating spatio-temporal context and pose constraints for full surround scene understanding.

Findings

Outperforms strong baselines on benchmark datasets.

Produces dense, consistent, and scale-aware 360-degree point clouds.

Introduces a new scale-consistent evaluation metric for multi-camera setups.

Abstract

Self-supervised monocular depth and ego-motion estimation is a promising approach to replace or supplement expensive depth sensors such as LiDAR for robotics applications like autonomous driving. However, most research in this area focuses on a single monocular camera or stereo pairs that cover only a fraction of the scene around the vehicle. In this work, we extend monocular self-supervised depth and ego-motion estimation to large-baseline multi-camera rigs. Using generalized spatio-temporal contexts, pose consistency constraints, and carefully designed photometric loss masking, we learn a single network generating dense, consistent, and scale-aware point clouds that cover the same full surround 360 degree field of view as a typical LiDAR scanner. We also propose a new scale-consistent evaluation metric more suitable to multi-camera settings. Experiments on two challenging benchmarks illustrate the benefits of our approach over strong baselines.