Monocular 3D Object Detection with LiDAR Guided Semi Supervised Active Learning

TL;DR

This paper introduces MonoLiG, a semi-supervised active learning framework for monocular 3D object detection that effectively utilizes LiDAR data for guiding data selection and training, improving accuracy and reducing labeling costs.

Contribution

The paper presents a novel framework that combines LiDAR guidance with semi-supervised active learning for monocular 3D detection, including a data noise-based weighting and sensor consistency-based sample selection.

Findings

Outperforms state-of-the-art active learning methods in labeling efficiency.

Achieves top results on KITTI 3D and BEV detection benchmarks.

Improves BEV AP by 2.02 on KITTI dataset.

Abstract

We propose a novel semi-supervised active learning (SSAL) framework for monocular 3D object detection with LiDAR guidance (MonoLiG), which leverages all modalities of collected data during model development. We utilize LiDAR to guide the data selection and training of monocular 3D detectors without introducing any overhead in the inference phase. During training, we leverage the LiDAR teacher, monocular student cross-modal framework from semi-supervised learning to distill information from unlabeled data as pseudo-labels. To handle the differences in sensor characteristics, we propose a data noise-based weighting mechanism to reduce the effect of propagating noise from LiDAR modality to monocular. For selecting which samples to label to improve the model performance, we propose a sensor consistency-based selection score that is also coherent with the training objective. Extensive experimental results on KITTI and Waymo datasets verify the effectiveness of our proposed framework. In particular, our selection strategy consistently outperforms state-of-the-art active learning baselines, yielding up to 17% better saving rate in labeling costs. Our training strategy attains the top place in KITTI 3D and birds-eye-view (BEV) monocular object detection official benchmarks by improving the BEV Average Precision (AP) by 2.02.