Once Detected, Never Lost: Surpassing Human Performance in Offline LiDAR based 3D Object Detection

TL;DR

This paper introduces CTRL, a track-centric offline LiDAR-based 3D object detection system that leverages temporal coherence to surpass human annotation accuracy and previous state-of-the-art methods.

Contribution

The paper proposes a novel track-centric detection framework with bidirectional tracking and learning modules, significantly improving detection performance.

Findings

Surpasses human annotation accuracy in LiDAR 3D detection

Outperforms previous state-of-the-art methods on Waymo dataset

Achieves high detection performance without model ensemble

Abstract

This paper aims for high-performance offline LiDAR-based 3D object detection. We first observe that experienced human annotators annotate objects from a track-centric perspective. They first label the objects with clear shapes in a track, and then leverage the temporal coherence to infer the annotations of obscure objects. Drawing inspiration from this, we propose a high-performance offline detector in a track-centric perspective instead of the conventional object-centric perspective. Our method features a bidirectional tracking module and a track-centric learning module. Such a design allows our detector to infer and refine a complete track once the object is detected at a certain moment. We refer to this characteristic as "onCe detecTed, neveR Lost" and name the proposed system CTRL. Extensive experiments demonstrate the remarkable performance of our method, surpassing the human-level annotating accuracy and the previous state-of-the-art methods in the highly competitive Waymo Open Dataset without model ensemble. The code will be made publicly available at https://github.com/tusen-ai/SST.