Dexterous In-hand Manipulation by Guiding Exploration with Simple Sub-skill Controllers

TL;DR

This paper introduces a framework that enhances sample efficiency in learning complex in-hand manipulation skills by guiding exploration with simple, domain-informed sub-skill controllers, enabling learning without reset distributions.

Contribution

It presents a novel exploration guidance method using sub-skill controllers to improve learning efficiency in dexterous in-hand manipulation tasks.

Findings

Improved sample efficiency in simulation-based learning.

Successful learning of finger-gaiting without reset distributions.

First demonstration of such skills guided by sub-skill controllers.

Abstract

Recently, reinforcement learning has led to dexterous manipulation skills of increasing complexity. Nonetheless, learning these skills in simulation still exhibits poor sample-efficiency which stems from the fact these skills are learned from scratch without the benefit of any domain expertise. In this work, we aim to improve the sample efficiency of learning dexterous in-hand manipulation skills using controllers available via domain knowledge. To this end, we design simple sub-skill controllers and demonstrate improved sample efficiency using a framework that guides exploration toward relevant state space by following actions from these controllers. We are the first to demonstrate learning hard-to-explore finger-gaiting in-hand manipulation skills without the use of an exploratory reset distribution. Video results can be found at https://roamlab.github.io/vge