CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds

TL;DR

CAGroup3D introduces a class-aware grouping strategy and a sparse convolutional RoI pooling module to enhance 3D object detection on point clouds, achieving state-of-the-art results efficiently.

Contribution

The paper proposes a novel class-aware local grouping and a sparse convolutional RoI pooling method for improved 3D detection accuracy.

Findings

Achieves +3.6% mAP on ScanNet V2

Achieves +2.6% mAP on SUN RGB-D

Efficient memory and computation usage

Abstract

We present a novel two-stage fully sparse convolutional 3D object detection framework, named CAGroup3D. Our proposed method first generates some high-quality 3D proposals by leveraging the class-aware local group strategy on the object surface voxels with the same semantic predictions, which considers semantic consistency and diverse locality abandoned in previous bottom-up approaches. Then, to recover the features of missed voxels due to incorrect voxel-wise segmentation, we build a fully sparse convolutional RoI pooling module to directly aggregate fine-grained spatial information from backbone for further proposal refinement. It is memory-and-computation efficient and can better encode the geometry-specific features of each 3D proposal. Our model achieves state-of-the-art 3D detection performance with remarkable gains of +\textit{3.6\%} on ScanNet V2 and +\textit{2.6}\% on SUN RGB-D in term of mAP@0.25. Code will be available at https://github.com/Haiyang-W/CAGroup3D.