Multi-Camera Collaborative Depth Prediction via Consistent Structure Estimation

TL;DR

This paper introduces a multi-camera depth prediction method that maintains structure consistency without requiring large overlaps, using a basis formulation and iterative refinement driven by a consistency loss, outperforming existing methods.

Contribution

The proposed approach enables multi-camera depth estimation without large overlaps, ensuring structure consistency through a novel basis formulation and iterative refinement with a consistency loss.

Findings

Outperforms existing methods on DDAD and NuScenes datasets.

Maintains structure consistency across cameras without large overlaps.

Effectively propagates overlapping area information across depth maps.

Abstract

Depth map estimation from images is an important task in robotic systems. Existing methods can be categorized into two groups including multi-view stereo and monocular depth estimation. The former requires cameras to have large overlapping areas and sufficient baseline between cameras, while the latter that processes each image independently can hardly guarantee the structure consistency between cameras. In this paper, we propose a novel multi-camera collaborative depth prediction method that does not require large overlapping areas while maintaining structure consistency between cameras. Specifically, we formulate the depth estimation as a weighted combination of depth basis, in which the weights are updated iteratively by a refinement network driven by the proposed consistency loss. During the iterative update, the results of depth estimation are compared across cameras and the information of overlapping areas is propagated to the whole depth maps with the help of basis formulation. Experimental results on DDAD and NuScenes datasets demonstrate the superior performance of our method.