StruMamba3D: Exploring Structural Mamba for Self-supervised Point Cloud Representation Learning

TL;DR

StruMamba3D introduces a novel self-supervised learning approach for point clouds that preserves spatial dependencies and improves robustness to input length variations, achieving state-of-the-art results on standard benchmarks.

Contribution

It proposes a new paradigm combining spatial states and a sequence length-adaptive strategy to enhance SSM-based point cloud learning.

Findings

Achieves 95.1% accuracy on ModelNet40

Attains 92.75% accuracy on ScanObjectNN

Outperforms existing methods on four downstream tasks

Abstract

Recently, Mamba-based methods have demonstrated impressive performance in point cloud representation learning by leveraging State Space Model (SSM) with the efficient context modeling ability and linear complexity. However, these methods still face two key issues that limit the potential of SSM: Destroying the adjacency of 3D points during SSM processing and failing to retain long-sequence memory as the input length increases in downstream tasks. To address these issues, we propose StruMamba3D, a novel paradigm for self-supervised point cloud representation learning. It enjoys several merits. First, we design spatial states and use them as proxies to preserve spatial dependencies among points. Second, we enhance the SSM with a state-wise update strategy and incorporate a lightweight convolution to facilitate interactions between spatial states for efficient structure modeling. Third, our method reduces the sensitivity of pre-trained Mamba-based models to varying input lengths by introducing a sequence length-adaptive strategy. Experimental results across four downstream tasks showcase the superior performance of our method. In addition, our method attains the SOTA 95.1% accuracy on ModelNet40 and 92.75% accuracy on the most challenging split of ScanObjectNN without voting strategy.