DM3D: Deformable Mamba via Offset-Guided Differentiable Scanning for Point Cloud Understanding

TL;DR

DM3D introduces an adaptive, deformable serialization method for point cloud understanding, improving structural awareness and achieving state-of-the-art results across multiple tasks.

Contribution

The paper proposes DM3D, a deformable Mamba architecture with offset-guided differentiable scanning for adaptive point cloud serialization.

Findings

DM3D achieves state-of-the-art results on classification tasks.

DM3D performs competitively on few-shot learning.

DM3D excels in part segmentation benchmarks.

Abstract

State Space Models (SSMs) show significant potential for long-sequence modeling, but their reliance on input order conflicts with the irregular nature of point clouds. Existing approaches often rely on predefined serialization schemes whose fixed scanning patterns cannot adapt to diverse geometric structures. To address this limitation, we propose DM3D, a deformable Mamba architecture for point cloud understanding. Specifically, DM3D introduces an offset-guided differentiable scanning mechanism that jointly performs resampling and reordering. Deformable Spatial Resampling (DSR) enhances structural awareness by adaptively resampling local features, while the Gaussian-based Differentiable Reordering (GDR) enables end-to-end optimization of the serialization order. We further introduce a Continuity-Aware State Update (CASU) mechanism that modulates the state update based on local geometric continuity. In addition, a Tri-Path Fusion module facilitates complementary interactions among different SSM branches. Together, these designs enable structure-adaptive serialization for point clouds. Extensive experiments on benchmark datasets show that DM3D achieves state-of-the-art or highly competitive results on classification, few-shot learning, and part segmentation tasks, validating the effectiveness of adaptive serialization for point cloud understanding.