SparseDet: Towards End-to-End 3D Object Detection

TL;DR

SparseDet introduces an end-to-end 3D object detection method using learnable proposals and transformers, eliminating the need for dense candidate generation and post-processing, achieving high accuracy and efficiency.

Contribution

The paper presents SparseDet, a novel sparse, end-to-end 3D detection framework that replaces dense candidate methods with learnable proposals and transformer architecture.

Findings

Achieves competitive detection accuracy.

Runs at 34.5 FPS.

Eliminates post-processing steps.

Abstract

In this paper, we propose SparseDet for end-to-end 3D object detection from point cloud. Existing works on 3D object detection rely on dense object candidates over all locations in a 3D or 2D grid following the mainstream methods for object detection in 2D images. However, this dense paradigm requires expertise in data to fulfill the gap between label and detection. As a new detection paradigm, SparseDet maintains a fixed set of learnable proposals to represent latent candidates and directly perform classification and localization for 3D objects through stacked transformers. It demonstrates that effective 3D object detection can be achieved with none of post-processing such as redundant removal and non-maximum suppression. With a properly designed network, SparseDet achieves highly competitive detection accuracy while running with a more efficient speed of 34.5 FPS. We believe this end-to-end paradigm of SparseDet will inspire new thinking on the sparsity of 3D object detection.