From Simple to Complex Skills: The Case of In-Hand Object Reorientation

TL;DR

This paper introduces a hierarchical policy system that leverages low-level skills and a pose estimator to improve in-hand object reorientation, enhancing robustness and sim-to-real transfer for complex manipulation tasks.

Contribution

The work presents a hierarchical policy framework and a generalizable pose estimator that together enable more robust and transferable in-hand object reorientation skills.

Findings

Hierarchical policy improves robustness to out-of-distribution changes.

System successfully reorients symmetrical and textureless objects.

Enhanced sim-to-real transfer performance.

Abstract

Learning policies in simulation and transferring them to the real world has become a promising approach in dexterous manipulation. However, bridging the sim-to-real gap for each new task requires substantial human effort, such as careful reward engineering, hyperparameter tuning, and system identification. In this work, we present a system that leverages low-level skills to address these challenges for more complex tasks. Specifically, we introduce a hierarchical policy for in-hand object reorientation based on previously acquired rotation skills. This hierarchical policy learns to select which low-level skill to execute based on feedback from both the environment and the low-level skill policies themselves. Compared to learning from scratch, the hierarchical policy is more robust to out-of-distribution changes and transfers easily from simulation to real-world environments. Additionally, we propose a generalizable object pose estimator that uses proprioceptive information, low-level skill predictions, and control errors as inputs to estimate the object pose over time. We demonstrate that our system can reorient objects, including symmetrical and textureless ones, to a desired pose.