Offboard 3D Object Detection from Point Cloud Sequences

TL;DR

This paper introduces a novel offboard 3D object detection pipeline using point cloud sequences, achieving high-quality labels suitable for offboard perception tasks, surpassing existing onboard detectors.

Contribution

The paper proposes a new offboard detection method leveraging temporal point cloud data and object-centric refinement, enabling high-quality 3D labeling for perception applications.

Findings

Significant performance improvements over state-of-the-art onboard detectors

Performance comparable to human labeling verified by a human label study

Effective use of auto labels for semi-supervised learning

Abstract

While current 3D object recognition research mostly focuses on the real-time, onboard scenario, there are many offboard use cases of perception that are largely under-explored, such as using machines to automatically generate high-quality 3D labels. Existing 3D object detectors fail to satisfy the high-quality requirement for offboard uses due to the limited input and speed constraints. In this paper, we propose a novel offboard 3D object detection pipeline using point cloud sequence data. Observing that different frames capture complementary views of objects, we design the offboard detector to make use of the temporal points through both multi-frame object detection and novel object-centric refinement models. Evaluated on the Waymo Open Dataset, our pipeline named 3D Auto Labeling shows significant gains compared to the state-of-the-art onboard detectors and our offboard baselines. Its performance is even on par with human labels verified through a human label study. Further experiments demonstrate the application of auto labels for semi-supervised learning and provide extensive analysis to validate various design choices.