RBGNet: Ray-based Grouping for 3D Object Detection

TL;DR

RBGNet introduces a ray-based feature grouping and foreground biased sampling to improve 3D object detection accuracy from point clouds, leveraging surface geometry for better shape representation and detection performance.

Contribution

The paper proposes a novel ray-based grouping module and a foreground biased sampling strategy, enhancing 3D detection by utilizing surface geometry and focused point sampling.

Findings

Achieves state-of-the-art results on ScanNet V2 and SUN RGB-D datasets.

Significant performance improvements over previous methods.

Effective surface-aware feature aggregation enhances 3D box prediction.

Abstract

As a fundamental problem in computer vision, 3D object detection is experiencing rapid growth. To extract the point-wise features from the irregularly and sparsely distributed points, previous methods usually take a feature grouping module to aggregate the point features to an object candidate. However, these methods have not yet leveraged the surface geometry of foreground objects to enhance grouping and 3D box generation. In this paper, we propose the RBGNet framework, a voting-based 3D detector for accurate 3D object detection from point clouds. In order to learn better representations of object shape to enhance cluster features for predicting 3D boxes, we propose a ray-based feature grouping module, which aggregates the point-wise features on object surfaces using a group of determined rays uniformly emitted from cluster centers. Considering the fact that foreground points are more meaningful for box estimation, we design a novel foreground biased sampling strategy in downsample process to sample more points on object surfaces and further boost the detection performance. Our model achieves state-of-the-art 3D detection performance on ScanNet V2 and SUN RGB-D with remarkable performance gains. Code will be available at https://github.com/Haiyang-W/RBGNet.