Exploring Geometric Consistency for Monocular 3D Object Detection

TL;DR

This paper addresses the challenge of geometric inconsistency in monocular 3D object detection by analyzing failure modes and proposing four geometry-aware data augmentation techniques that improve detection accuracy and generalization.

Contribution

The paper introduces four novel geometry-aware data augmentation methods that enhance geometric consistency in monocular 3D detection models, leading to state-of-the-art results.

Findings

Improved detection accuracy on KITTI and nuScenes benchmarks.

Enhanced semi-supervised training and cross-dataset generalization.

Demonstrated importance of maintaining geometric consistency during augmentation.

Abstract

This paper investigates the geometric consistency for monocular 3D object detection, which suffers from the ill-posed depth estimation. We first conduct a thorough analysis to reveal how existing methods fail to consistently localize objects when different geometric shifts occur. In particular, we design a series of geometric manipulations to diagnose existing detectors and then illustrate their vulnerability to consistently associate the depth with object apparent sizes and positions. To alleviate this issue, we propose four geometry-aware data augmentation approaches to enhance the geometric consistency of the detectors. We first modify some commonly used data augmentation methods for 2D images so that they can maintain geometric consistency in 3D spaces. We demonstrate such modifications are important. In addition, we propose a 3D-specific image perturbation method that employs the camera movement. During the augmentation process, the camera system with the corresponding image is manipulated, while the geometric visual cues for depth recovery are preserved. We show that by using the geometric consistency constraints, the proposed augmentation techniques lead to improvements on the KITTI and nuScenes monocular 3D detection benchmarks with state-of-the-art results. In addition, we demonstrate that the augmentation methods are well suited for semi-supervised training and cross-dataset generalization.