On Policy Learning Robust to Irreversible Events: An Application to Robotic In-Hand Manipulation

TL;DR

This paper introduces a reinforcement learning approach combined with tactile reactive control for robotic in-hand manipulation, effectively reducing irreversible slipping events and improving task success in low-cost prosthetic hands.

Contribution

It presents a novel integration of visual-based reinforcement learning with tactile reactive control to prevent irreversible slips during in-hand manipulation tasks.

Findings

Reduced object slipping during learning

Successful reorientation of objects using fingers

Reactive control is essential for avoiding irreversible events

Abstract

In this letter, we present an approach for learning in-hand manipulation skills with a low-cost, underactuated prosthetic hand in the presence of irreversible events. Our approach combines reinforcement learning based on visual perception with low-level reactive control based on tactile perception, which aims to avoid slipping. The objective of the reinforcement learning level consists not only in fulfilling the in-hand manipulation goal, but also in minimizing the intervention of the tactile reactive control. This way, the occurrence of object slipping during the learning procedure, which we consider an irreversible event, is significantly reduced. When an irreversible event occurs, the learning process is considered failed. We show the performance in two tasks, which consist in reorienting a cup and a bottle only using the fingers. The experimental results show that the proposed architecture allows reaching the goal in the Cartesian space and reduces significantly the occurrence of object slipping during the learning procedure. Moreover, without the proposed synergy between reactive control and reinforcement learning it was not possible to avoid irreversible events and, therefore, to learn the task.