Deformation-based In-Context Learning for Point Cloud Understanding

TL;DR

DeformPIC introduces a deformation-based approach for point cloud in-context learning, enabling explicit geometric reasoning and outperforming prior masked reconstruction methods across multiple tasks and out-of-domain benchmarks.

Contribution

The paper proposes DeformPIC, a novel deformation-based framework that addresses limitations of masked point modeling in point cloud ICL, improving geometric reasoning and generalization.

Findings

DeformPIC reduces Chamfer Distance by 1.6, 1.8, and 4.7 points on reconstruction, denoising, and registration tasks.

DeformPIC outperforms previous methods on a new out-of-domain benchmark.

Extensive experiments validate the effectiveness of DeformPIC across multiple point cloud tasks.

Abstract

Recent advances in point cloud In-Context Learning (ICL) have demonstrated strong multitask capabilities. Existing approaches typically adopt a Masked Point Modeling (MPM)-based paradigm for point cloud ICL. However, MPM-based methods directly predict the target point cloud from masked tokens without leveraging geometric priors, requiring the model to infer spatial structure and geometric details solely from token-level correlations via transformers. Additionally, these methods suffer from a training-inference objective mismatch, as the model learns to predict the target point cloud using target-side information that is unavailable at inference time. To address these challenges, we propose DeformPIC, a deformation-based framework for point cloud ICL. Unlike existing approaches that rely on masked reconstruction, DeformPIC learns to deform the query point cloud under task-specific guidance from prompts, enabling explicit geometric reasoning and consistent objectives. Extensive experiments demonstrate that DeformPIC consistently outperforms previous state-of-the-art methods, achieving reductions of 1.6, 1.8, and 4.7 points in average Chamfer Distance on reconstruction, denoising, and registration tasks, respectively. Furthermore, we introduce a new out-of-domain benchmark to evaluate generalization across unseen data distributions, where DeformPIC achieves state-of-the-art performance.