Active Pose Refinement for Textureless Shiny Objects using the Structured Light Camera

TL;DR

This paper introduces an active vision framework that refines 6D pose estimation of shiny, textureless objects using structured light cameras, by optimizing pose refinement and next-best-view prediction to handle specular reflections.

Contribution

It presents a novel optimization-based pose refinement module and a next-best-view prediction strategy that predicts measurement uncertainties, improving pose accuracy with fewer viewpoints.

Findings

Outperforms traditional ICP-based methods in pose refinement.

Achieves high pose accuracy with fewer viewpoints.

Effectively handles shiny, textureless objects with structured light cameras.

Abstract

6D pose estimation of textureless shiny objects has become an essential problem in many robotic applications. Many pose estimators require high-quality depth data, often measured by structured light cameras. However, when objects have shiny surfaces (e.g., metal parts), these cameras fail to sense complete depths from a single viewpoint due to the specular reflection, resulting in a significant drop in the final pose accuracy. To mitigate this issue, we present a complete active vision framework for 6D object pose refinement and next-best-view prediction. Specifically, we first develop an optimization-based pose refinement module for the structured light camera. Our system then selects the next best camera viewpoint to collect depth measurements by minimizing the predicted uncertainty of the object pose. Compared to previous approaches, we additionally predict measurement uncertainties of future viewpoints by online rendering, which significantly improves the next-best-view prediction performance. We test our approach on the challenging real-world ROBI dataset. The results demonstrate that our pose refinement method outperforms the traditional ICP-based approach when given the same input depth data, and our next-best-view strategy can achieve high object pose accuracy with significantly fewer viewpoints than the heuristic-based policies.