MICAS: Multi-grained In-Context Adaptive Sampling for 3D Point Cloud Processing

TL;DR

MICAS introduces a novel multi-grained adaptive sampling framework for in-context learning in 3D point cloud processing, addressing task sensitivity issues and significantly improving performance across multiple PCP tasks.

Contribution

This work is the first to develop adaptive sampling strategies tailored specifically for point clouds within an in-context learning framework.

Findings

Achieves 4.1% improvement in part segmentation accuracy.

Outperforms existing methods across various PCP tasks.

Effectively handles inter-task and intra-task sensitivity issues.

Abstract

Point cloud processing (PCP) encompasses tasks like reconstruction, denoising, registration, and segmentation, each often requiring specialized models to address unique task characteristics. While in-context learning (ICL) has shown promise across tasks by using a single model with task-specific demonstration prompts, its application to PCP reveals significant limitations. We identify inter-task and intra-task sensitivity issues in current ICL methods for PCP, which we attribute to inflexible sampling strategies lacking context adaptation at the point and prompt levels. To address these challenges, we propose MICAS, an advanced ICL framework featuring a multi-grained adaptive sampling mechanism tailored for PCP. MICAS introduces two core components: task-adaptive point sampling, which leverages inter-task cues for point-level sampling, and query-specific prompt sampling, which selects optimal prompts per query to mitigate intra-task sensitivity. To our knowledge, this is the first approach to introduce adaptive sampling tailored to the unique requirements of point clouds within an ICL framework. Extensive experiments show that MICAS not only efficiently handles various PCP tasks but also significantly outperforms existing methods. Notably, it achieves a remarkable $4.1\%$ improvement in the part segmentation task and delivers consistent gains across various PCP applications.