STS: Surround-view Temporal Stereo for Multi-view 3D Detection

TL;DR

This paper introduces Surround-view Temporal Stereo (STS), a novel method that combines temporal stereo matching across multiple camera views and time to improve depth estimation for multi-view 3D object detection.

Contribution

The paper proposes a new surround-view temporal stereo technique that leverages multi-view and temporal geometric correspondence to enhance depth learning in 3D detection.

Findings

STS significantly improves detection accuracy for medium and long-distance objects.

On nuScenes, STS boosts mAP by 2.6% and NDS by 1.4%.

The method shows consistent gains with larger backbones and higher resolutions.

Abstract

Learning accurate depth is essential to multi-view 3D object detection. Recent approaches mainly learn depth from monocular images, which confront inherent difficulties due to the ill-posed nature of monocular depth learning. Instead of using a sole monocular depth method, in this work, we propose a novel Surround-view Temporal Stereo (STS) technique that leverages the geometry correspondence between frames across time to facilitate accurate depth learning. Specifically, we regard the field of views from all cameras around the ego vehicle as a unified view, namely surroundview, and conduct temporal stereo matching on it. The resulting geometrical correspondence between different frames from STS is utilized and combined with the monocular depth to yield final depth prediction. Comprehensive experiments on nuScenes show that STS greatly boosts 3D detection ability, notably for medium and long distance objects. On BEVDepth with ResNet-50 backbone, STS improves mAP and NDS by 2.6% and 1.4%, respectively. Consistent improvements are observed when using a larger backbone and a larger image resolution, demonstrating its effectiveness