Towards Real-World Category-level Articulation Pose Estimation

TL;DR

This paper introduces a new real-world setting for category-level articulation pose estimation, along with datasets and a framework that handle multiple instances and varied kinematic structures in complex environments.

Contribution

It proposes the CAPER task setting, creates a comprehensive dataset and model repository, and develops ReArtNOCS for efficient multi-instance pose estimation in real-world scenarios.

Findings

ReArtNOCS achieves good performance on CAPER and CAPE datasets.

The datasets support varied kinematic structures and multiple instances.

The framework enables efficient part-level pose estimation in complex scenes.

Abstract

Human life is populated with articulated objects. Current Category-level Articulation Pose Estimation (CAPE) methods are studied under the single-instance setting with a fixed kinematic structure for each category. Considering these limitations, we reform this problem setting for real-world environments and suggest a CAPE-Real (CAPER) task setting. This setting allows varied kinematic structures within a semantic category, and multiple instances to co-exist in an observation of real world. To support this task, we build an articulated model repository ReArt-48 and present an efficient dataset generation pipeline, which contains Fast Articulated Object Modeling (FAOM) and Semi-Authentic MixEd Reality Technique (SAMERT). Accompanying the pipeline, we build a large-scale mixed reality dataset ReArtMix and a real world dataset ReArtVal. We also propose an effective framework ReArtNOCS that exploits RGB-D input to estimate part-level pose for multiple instances in a single forward pass. Extensive experiments demonstrate that the proposed ReArtNOCS can achieve good performance on both CAPER and CAPE settings. We believe it could serve as a strong baseline for future research on the CAPER task.