SEED: A Simple and Effective 3D DETR in Point Clouds

TL;DR

SEED introduces a simple yet effective 3D DETR framework for point cloud object detection, utilizing dual query selection and deformable grid attention to improve performance on large-scale datasets.

Contribution

The paper proposes SEED, a novel 3D DETR method with dual query selection and deformable grid attention modules, addressing key challenges in 3D point cloud detection.

Findings

Achieves state-of-the-art results on Waymo and nuScenes datasets.

Demonstrates the effectiveness of DQS and DGA modules through ablation studies.

Outperforms existing 3D detection methods in accuracy and robustness.

Abstract

Recently, detection transformers (DETRs) have gradually taken a dominant position in 2D detection thanks to their elegant framework. However, DETR-based detectors for 3D point clouds are still difficult to achieve satisfactory performance. We argue that the main challenges are twofold: 1) How to obtain the appropriate object queries is challenging due to the high sparsity and uneven distribution of point clouds; 2) How to implement an effective query interaction by exploiting the rich geometric structure of point clouds is not fully explored. To this end, we propose a simple and effective 3D DETR method (SEED) for detecting 3D objects from point clouds, which involves a dual query selection (DQS) module and a deformable grid attention (DGA) module. More concretely, to obtain appropriate queries, DQS first ensures a high recall to retain a large number of queries by the predicted confidence scores and then further picks out high-quality queries according to the estimated quality scores. DGA uniformly divides each reference box into grids as the reference points and then utilizes the predicted offsets to achieve a flexible receptive field, allowing the network to focus on relevant regions and capture more informative features. Extensive ablation studies on DQS and DGA demonstrate its effectiveness. Furthermore, our SEED achieves state-of-the-art detection performance on both the large-scale Waymo and nuScenes datasets, illustrating the superiority of our proposed method. The code is available at https://github.com/happinesslz/SEED