Mamba Learns in Context: Structure-Aware Domain Generalization for Multi-Task Point Cloud Understanding

TL;DR

This paper introduces a structure-aware domain generalization framework for multi-task point cloud understanding, combining novel serialization, hierarchical modeling, and spectral alignment to improve structural fidelity across domains.

Contribution

It proposes a new SADG framework with structure-aware serialization, hierarchical domain modeling, and spectral graph alignment for robust multi-task point cloud analysis.

Findings

Outperforms state-of-the-art methods across multiple tasks

Improves structural fidelity in multi-task domain generalization

Demonstrates effectiveness on the new MP3DObject dataset

Abstract

While recent Transformer and Mamba architectures have advanced point cloud representation learning, they are typically developed for single-task or single-domain settings. Directly applying them to multi-task domain generalization (DG) leads to degraded performance. Transformers effectively model global dependencies but suffer from quadratic attention cost and lack explicit structural ordering, whereas Mamba offers linear-time recurrence yet often depends on coordinate-driven serialization, which is sensitive to viewpoint changes and missing regions, causing structural drift and unstable sequential modeling. In this paper, we propose Structure-Aware Domain Generalization (SADG), a Mamba-based In-Context Learning framework that preserves structural hierarchy across domains and tasks. We design structure-aware serialization (SAS) that generates transformation-invariant sequences using centroid-based topology and geodesic curvature continuity. We further devise hierarchical domain-aware modeling (HDM) that stabilizes cross-domain reasoning by consolidating intra-domain structure and fusing inter-domain relations. At test time, we introduce a lightweight spectral graph alignment (SGA) that shifts target features toward source prototypes in the spectral domain without updating model parameters, ensuring structure-preserving test-time feature shifting. In addition, we introduce MP3DObject, a real-scan object dataset for multi-task DG evaluation. Comprehensive experiments demonstrate that the proposed approach improves structural fidelity and consistently outperforms state-of-the-art methods across multiple tasks including reconstruction, denoising, and registration.