ZigzagPointMamba: Spatial-Semantic Mamba for Point Cloud Understanding

TL;DR

ZigzagPointMamba introduces a spatially continuous token sequencing and semantic-aware masking strategy to improve point cloud understanding in self-supervised learning, leading to better downstream task performance.

Contribution

It proposes ZigzagPointMamba with a zigzag scan path and Semantic-Siamese Masking Strategy to enhance spatial and semantic modeling in point cloud self-supervised learning.

Findings

Achieves 1.59% mIoU improvement on ShapeNetPart.

Improves classification accuracy on ModelNet40 by 0.4%.

Enhances ScanObjectNN subset accuracies by up to 1.22%.

Abstract

State Space models (SSMs) such as PointMamba enable efficient feature extraction for point cloud self-supervised learning with linear complexity, outperforming Transformers in computational efficiency. However, existing PointMamba-based methods depend on complex token ordering and random masking, which disrupt spatial continuity and local semantic correlations. We propose ZigzagPointMamba to tackle these challenges. The core of our approach is a simple zigzag scan path that globally sequences point cloud tokens, enhancing spatial continuity by preserving the proximity of spatially adjacent point tokens. Nevertheless, random masking undermines local semantic modeling in self-supervised learning. To address this, we introduce a Semantic-Siamese Masking Strategy (SMS), which masks semantically similar tokens to facilitate reconstruction by integrating local features of original and similar tokens. This overcomes the dependence on isolated local features and enables robust global semantic modeling. Our pre-trained ZigzagPointMamba weights significantly improve downstream tasks, achieving a 1.59% mIoU gain on ShapeNetPart for part segmentation, a 0.4% higher accuracy on ModelNet40 for classification, and 0.19%, 1.22%, and 0.72% higher accuracies respectively for the classification tasks on the OBJ-BG, OBJ-ONLY, and PB-T50-RS subsets of ScanObjectNN.