Pamba: Enhancing Global Interaction in Point Clouds via State Space Model

TL;DR

Pamba introduces a novel SSM-based architecture for 3D point cloud segmentation that achieves state-of-the-art results with linear complexity, enabling efficient global and local modeling of complex scenes.

Contribution

It proposes Pamba, a new architecture combining state space models with multi-path serialization and ConvMamba blocks for improved global and local point cloud modeling.

Findings

Achieves state-of-the-art results on multiple 3D segmentation benchmarks.

Demonstrates linear complexity in processing large point clouds.

Validates effectiveness through extensive experiments.

Abstract

Transformers have demonstrated impressive results for 3D point cloud semantic segmentation. However, the quadratic complexity of transformer makes computation costs high, limiting the number of points that can be processed simultaneously and impeding the modeling of long-range dependencies between objects in a single scene. Drawing inspiration from the great potential of recent state space models (SSM) for long sequence modeling, we introduce Mamba, an SSM-based architecture, to the point cloud domain and propose Pamba, a novel architecture with strong global modeling capability under linear complexity. Specifically, to make the disorderness of point clouds fit in with the causal nature of Mamba, we propose a multi-path serialization strategy applicable to point clouds. Besides, we propose the ConvMamba block to compensate for the shortcomings of Mamba in modeling local geometries and in unidirectional modeling. Pamba obtains state-of-the-art results on several 3D point cloud segmentation tasks, including ScanNet v2, ScanNet200, S3DIS and nuScenes, while its effectiveness is validated by extensive experiments.