DSGN: Deep Stereo Geometry Network for 3D Object Detection

TL;DR

DSGN introduces a novel stereo-based 3D object detection method that uses a differentiable volumetric representation to jointly estimate depth and detect objects, significantly narrowing the performance gap with LiDAR-based methods.

Contribution

The paper presents the first end-to-end stereo-based 3D detection pipeline using a 3D geometric volume for improved accuracy and efficiency.

Findings

Outperforms previous stereo-based detectors by about 10 AP.

Achieves comparable performance with LiDAR-based methods on KITTI.

Provides a simple, effective one-stage stereo detection framework.

Abstract

Most state-of-the-art 3D object detectors heavily rely on LiDAR sensors because there is a large performance gap between image-based and LiDAR-based methods. It is caused by the way to form representation for the prediction in 3D scenarios. Our method, called Deep Stereo Geometry Network (DSGN), significantly reduces this gap by detecting 3D objects on a differentiable volumetric representation -- 3D geometric volume, which effectively encodes 3D geometric structure for 3D regular space. With this representation, we learn depth information and semantic cues simultaneously. For the first time, we provide a simple and effective one-stage stereo-based 3D detection pipeline that jointly estimates the depth and detects 3D objects in an end-to-end learning manner. Our approach outperforms previous stereo-based 3D detectors (about 10 higher in terms of AP) and even achieves comparable performance with several LiDAR-based methods on the KITTI 3D object detection leaderboard. Our code is publicly available at https://github.com/chenyilun95/DSGN.