APriCoT: Action Primitives based on Contact-state Transition for In-Hand Tool Manipulation

TL;DR

This paper introduces APriCoT, a deep reinforcement learning approach that decomposes in-hand tool manipulation into contact-state transition primitives, improving learning efficiency and robustness in rotating objects within the hand.

Contribution

The paper proposes a novel contact-state transition-based primitive decomposition method, enhancing sample efficiency and robustness in in-hand manipulation tasks.

Findings

Successfully rotated objects within the hand to desired positions.

Achieved robustness to variations in object shape.

Outperformed existing methods in grasp achievement.

Abstract

In-hand tool manipulation is an operation that not only manipulates a tool within the hand (i.e., in-hand manipulation) but also achieves a grasp suitable for a task after the manipulation. This study aims to achieve an in-hand tool manipulation skill through deep reinforcement learning. The difficulty of learning the skill arises because this manipulation requires (A) exploring long-term contact-state changes to achieve the desired grasp and (B) highly-varied motions depending on the contact-state transition. (A) leads to a sparsity of a reward on a successful grasp, and (B) requires an RL agent to explore widely within the state-action space to learn highly-varied actions, leading to sample inefficiency. To address these issues, this study proposes Action Primitives based on Contact-state Transition (APriCoT). APriCoT decomposes the manipulation into short-term action primitives by describing the operation as a contact-state transition based on three action representations (detach, crossover, attach). In each action primitive, fingers are required to perform short-term and similar actions. By training a policy for each primitive, we can mitigate the issues from (A) and (B). This study focuses on a fundamental operation as an example of in-hand tool manipulation: rotating an elongated object grasped with a precision grasp by half a turn to achieve the initial grasp. Experimental results demonstrated that ours succeeded in both the rotation and the achievement of the desired grasp, unlike existing studies. Additionally, it was found that the policy was robust to changes in object shape.