ALPI: Auto-Labeller with Proxy Injection for 3D Object Detection using 2D Labels Only

TL;DR

This paper introduces ALPI, a weakly supervised 3D object detection method that uses only 2D labels and size priors, employing proxy objects and pseudo-labeling to achieve competitive results.

Contribution

ALPI presents a novel approach for 3D detection using 2D annotations, proxy objects, and depth-invariant loss functions, reducing annotation costs.

Findings

Performs on-par or better than previous methods on KITTI Car detection.

Achieves near fully supervised performance on challenging classes.

Demonstrates robustness on the nuScenes dataset.

Abstract

3D object detection plays a crucial role in various applications such as autonomous vehicles, robotics and augmented reality. However, training 3D detectors requires a costly precise annotation, which is a hindrance to scaling annotation to large datasets. To address this challenge, we propose a weakly supervised 3D annotator that relies solely on 2D bounding box annotations from images, along with size priors. One major problem is that supervising a 3D detection model using only 2D boxes is not reliable due to ambiguities between different 3D poses and their identical 2D projection. We introduce a simple yet effective and generic solution: we build 3D proxy objects with annotations by construction and add them to the training dataset. Our method requires only size priors to adapt to new classes. To better align 2D supervision with 3D detection, our method ensures depth invariance with a novel expression of the 2D losses. Finally, to detect more challenging instances, our annotator follows an offline pseudo-labelling scheme which gradually improves its 3D pseudo-labels. Extensive experiments on the KITTI dataset demonstrate that our method not only performs on-par or above previous works on the Car category, but also achieves performance close to fully supervised methods on more challenging classes. We further demonstrate the effectiveness and robustness of our method by being the first to experiment on the more challenging nuScenes dataset. We additionally propose a setting where weak labels are obtained from a 2D detector pre-trained on MS-COCO instead of human annotations. The code is available at https://github.com/CEA-LIST/ALPI