ViHOPE: Visuotactile In-Hand Object 6D Pose Estimation with Shape Completion

TL;DR

ViHOPE introduces a visuotactile shape completion framework that significantly enhances 6D object pose estimation accuracy in robotic in-hand manipulation by explicitly completing object shapes and jointly optimizing shape and pose.

Contribution

The paper presents a novel visuotactile shape completion module using a conditional GAN, improving 6D pose estimation accuracy over prior methods by explicitly completing object shapes.

Findings

Outperforms state-of-the-art in shape completion by 265% IoU and 88% Chamfer Distance reduction.

Reduces position and angular errors in pose estimation by 35% and 64%.

Demonstrates robustness to sim-to-real transfer on a real robot platform.

Abstract

In this letter, we introduce ViHOPE, a novel framework for estimating the 6D pose of an in-hand object using visuotactile perception. Our key insight is that the accuracy of the 6D object pose estimate can be improved by explicitly completing the shape of the object. To this end, we introduce a novel visuotactile shape completion module that uses a conditional Generative Adversarial Network to complete the shape of an in-hand object based on volumetric representation. This approach improves over prior works that directly regress visuotactile observations to a 6D pose. By explicitly completing the shape of the in-hand object and jointly optimizing the shape completion and pose estimation tasks, we improve the accuracy of the 6D object pose estimate. We train and test our model on a synthetic dataset and compare it with the state-of-the-art. In the visuotactile shape completion task, we outperform the state-of-the-art by 265% using the Intersection of Union metric and achieve 88% lower Chamfer Distance. In the visuotactile pose estimation task, we present results that suggest our framework reduces position and angular errors by 35% and 64%, respectively. Furthermore, we ablate our framework to confirm the gain on the 6D object pose estimate from explicitly completing the shape. Ultimately, we show that our framework produces models that are robust to sim-to-real transfer on a real-world robot platform.