Category-Agnostic 6D Pose Estimation with Conditional Neural Processes

TL;DR

This paper introduces a category-agnostic meta-learning approach using neural processes and graph neural networks for 6D pose estimation, demonstrating strong generalization to unseen objects and occluded scenes.

Contribution

It proposes a novel neural process-based meta-learning framework with a geometry-aware decoder for cross-category 6D pose estimation, enabling generalization to new object categories.

Findings

Performs well on unseen objects with different shapes and appearances

Shows robustness in occluded scenes despite training without occlusion

Achieves first cross-category level 6D pose estimation

Abstract

We present a novel meta-learning approach for 6D pose estimation on unknown objects. In contrast to ``instance-level" and ``category-level" pose estimation methods, our algorithm learns object representation in a category-agnostic way, which endows it with strong generalization capabilities across object categories. Specifically, we employ a neural process-based meta-learning approach to train an encoder to capture texture and geometry of an object in a latent representation, based on very few RGB-D images and ground-truth keypoints. The latent representation is then used by a simultaneously meta-trained decoder to predict the 6D pose of the object in new images. Furthermore, we propose a novel geometry-aware decoder for the keypoint prediction using a Graph Neural Network (GNN), which explicitly takes geometric constraints specific to each object into consideration. To evaluate our algorithm, extensive experiments are conducted on the \linemod dataset, and on our new fully-annotated synthetic datasets generated from Multiple Categories in Multiple Scenes (MCMS). Experimental results demonstrate that our model performs well on unseen objects with very different shapes and appearances. Remarkably, our model also shows robust performance on occluded scenes although trained fully on data without occlusion. To our knowledge, this is the first work exploring \textbf{cross-category level} 6D pose estimation.