SASA: Semantics-Augmented Set Abstraction for Point-based 3D Object Detection

TL;DR

This paper introduces SASA, a semantics-augmented set abstraction method that improves point-based 3D object detection by better identifying and retaining important foreground points, closing the gap with voxel-based methods.

Contribution

The paper proposes a novel set abstraction technique with a foreground segmentation module and semantics-guided sampling, enhancing point-based detection performance.

Findings

SASA improves detection accuracy on KITTI and nuScenes datasets.

It boosts various point-based detectors, achieving performance comparable to voxel-based methods.

Extensive experiments validate the effectiveness of SASA in 3D detection tasks.

Abstract

Although point-based networks are demonstrated to be accurate for 3D point cloud modeling, they are still falling behind their voxel-based competitors in 3D detection. We observe that the prevailing set abstraction design for down-sampling points may maintain too much unimportant background information that can affect feature learning for detecting objects. To tackle this issue, we propose a novel set abstraction method named Semantics-Augmented Set Abstraction (SASA). Technically, we first add a binary segmentation module as the side output to help identify foreground points. Based on the estimated point-wise foreground scores, we then propose a semantics-guided point sampling algorithm to help retain more important foreground points during down-sampling. In practice, SASA shows to be effective in identifying valuable points related to foreground objects and improving feature learning for point-based 3D detection. Additionally, it is an easy-to-plug-in module and able to boost various point-based detectors, including single-stage and two-stage ones. Extensive experiments on the popular KITTI and nuScenes datasets validate the superiority of SASA, lifting point-based detection models to reach comparable performance to state-of-the-art voxel-based methods.