Group-Free 3D Object Detection via Transformers

TL;DR

This paper introduces a transformer-based approach for 3D object detection from point clouds that eliminates the need for point grouping, leading to improved accuracy and state-of-the-art results on benchmark datasets.

Contribution

It proposes a novel direct detection method using attention mechanisms in transformers, removing the traditional point grouping step for better performance.

Findings

Achieves state-of-the-art results on ScanNet V2 and SUN RGB-D datasets.

Outperforms previous methods in 3D object detection accuracy.

Demonstrates the effectiveness of transformer-based feature aggregation in 3D detection.

Abstract

Recently, directly detecting 3D objects from 3D point clouds has received increasing attention. To extract object representation from an irregular point cloud, existing methods usually take a point grouping step to assign the points to an object candidate so that a PointNet-like network could be used to derive object features from the grouped points. However, the inaccurate point assignments caused by the hand-crafted grouping scheme decrease the performance of 3D object detection. In this paper, we present a simple yet effective method for directly detecting 3D objects from the 3D point cloud. Instead of grouping local points to each object candidate, our method computes the feature of an object from all the points in the point cloud with the help of an attention mechanism in the Transformers \cite{vaswani2017attention}, where the contribution of each point is automatically learned in the network training. With an improved attention stacking scheme, our method fuses object features in different stages and generates more accurate object detection results. With few bells and whistles, the proposed method achieves state-of-the-art 3D object detection performance on two widely used benchmarks, ScanNet V2 and SUN RGB-D. The code and models are publicly available at \url{https://github.com/zeliu98/Group-Free-3D}