Towards Stable 3D Object Detection

TL;DR

This paper introduces the Stability Index (SI) to evaluate 3D object detection stability in autonomous driving, and proposes Prediction Consistency Learning (PCL) to improve stability, demonstrating significant performance gains on the Waymo dataset.

Contribution

It presents a novel stability metric (SI) for 3D detectors and a training strategy (PCL) to enhance detection stability, addressing a previously overlooked aspect.

Findings

SI reveals new properties of object stability.

PCL improves SI scores significantly.

Achieved SI of 86.00 for vehicle detection.

Abstract

In autonomous driving, the temporal stability of 3D object detection greatly impacts the driving safety. However, the detection stability cannot be accessed by existing metrics such as mAP and MOTA, and consequently is less explored by the community. To bridge this gap, this work proposes Stability Index (SI), a new metric that can comprehensively evaluate the stability of 3D detectors in terms of confidence, box localization, extent, and heading. By benchmarking state-of-the-art object detectors on the Waymo Open Dataset, SI reveals interesting properties of object stability that have not been previously discovered by other metrics. To help models improve their stability, we further introduce a general and effective training strategy, called Prediction Consistency Learning (PCL). PCL essentially encourages the prediction consistency of the same objects under different timestamps and augmentations, leading to enhanced detection stability. Furthermore, we examine the effectiveness of PCL with the widely-used CenterPoint, and achieve a remarkable SI of 86.00 for vehicle class, surpassing the baseline by 5.48. We hope our work could serve as a reliable baseline and draw the community's attention to this crucial issue in 3D object detection. Codes will be made publicly available.