SGM3D: Stereo Guided Monocular 3D Object Detection

TL;DR

SGM3D introduces a stereo-guided monocular 3D detection framework that leverages stereo features and domain adaptation techniques to improve accuracy without relying on additional depth sensors.

Contribution

The paper proposes a novel stereo-guided monocular 3D detection method with multi-granularity domain adaptation and IoU-based object alignment, enhancing monocular detection performance.

Findings

Achieves state-of-the-art results on KITTI and Lyft datasets.

Effectively utilizes stereo features to guide monocular 3D detection.

Demonstrates robustness without extra depth sensors.

Abstract

Monocular 3D object detection aims to predict the object location, dimension and orientation in 3D space alongside the object category given only a monocular image. It poses a great challenge due to its ill-posed property which is critically lack of depth information in the 2D image plane. While there exist approaches leveraging off-the-shelve depth estimation or relying on LiDAR sensors to mitigate this problem, the dependence on the additional depth model or expensive equipment severely limits their scalability to generic 3D perception. In this paper, we propose a stereo-guided monocular 3D object detection framework, dubbed SGM3D, adapting the robust 3D features learned from stereo inputs to enhance the feature for monocular detection. We innovatively present a multi-granularity domain adaptation (MG-DA) mechanism to exploit the network's ability to generate stereo-mimicking features given only on monocular cues. Coarse BEV feature-level, as well as the fine anchor-level domain adaptation, are both leveraged for guidance in the monocular domain.In addition, we introduce an IoU matching-based alignment (IoU-MA) method for object-level domain adaptation between the stereo and monocular predictions to alleviate the mismatches while adopting the MG-DA. Extensive experiments demonstrate state-of-the-art results on KITTI and Lyft datasets.