Efficient 3D Perception on Multi-Sweep Point Cloud with Gumbel Spatial Pruning

TL;DR

This paper introduces Gumbel Spatial Pruning, a method that efficiently reduces redundancy in accumulated multi-sweep point clouds, enabling the use of more sweeps for improved outdoor 3D perception without extra computational costs.

Contribution

We propose a Gumbel Spatial Pruning layer that dynamically prunes redundant points in accumulated point clouds, allowing the use of more sweeps and enhancing perception accuracy.

Findings

Increased number of sweeps from 10 to 40 improves perception accuracy.

GSP layer maintains performance while reducing computational load.

Enhanced 3D detection and segmentation results on nuScenes dataset.

Abstract

This paper studies point cloud perception within outdoor environments. Existing methods face limitations in recognizing objects located at a distance or occluded, due to the sparse nature of outdoor point clouds. In this work, we observe a significant mitigation of this problem by accumulating multiple temporally consecutive point cloud sweeps, resulting in a remarkable improvement in perception accuracy. However, the computation cost also increases, hindering previous approaches from utilizing a large number of point cloud sweeps. To tackle this challenge, we find that a considerable portion of points in the accumulated point cloud is redundant, and discarding these points has minimal impact on perception accuracy. We introduce a simple yet effective Gumbel Spatial Pruning (GSP) layer that dynamically prunes points based on a learned end-to-end sampling. The GSP layer is decoupled from other network components and thus can be seamlessly integrated into existing point cloud network architectures. Without incurring additional computational overhead, we increase the number of point cloud sweeps from 10, a common practice, to as many as 40. Consequently, there is a significant enhancement in perception performance. For instance, in nuScenes 3D object detection and BEV map segmentation tasks, our pruning strategy improves several 3D perception baseline methods.