Learning Tactile Insertion in the Real World

TL;DR

This paper demonstrates that reinforcement learning with Dreamer-v3 can effectively utilize tactile sensing for robotic insertion tasks in both simulation and real-world settings, advancing tactile-based manipulation capabilities.

Contribution

It introduces an end-to-end reinforcement learning approach using Dreamer-v3 for tactile-based robotic insertion, with a self-resetting platform enabling extensive real-world training.

Findings

Dreamer-v3 successfully learns tactile insertion in simulation and real world.

Tactile feedback improves manipulation task performance.

The autonomous platform allows extensive real-world training without human intervention.

Abstract

Humans have exceptional tactile sensing capabilities, which they can leverage to solve challenging, partially observable tasks that cannot be solved from visual observation alone. Research in tactile sensing attempts to unlock this new input modality for robots. Lately, these sensors have become cheaper and, thus, widely available. At the same time, the question of how to integrate them into control loops is still an active area of research, with central challenges being partial observability and the contact-rich nature of manipulation tasks. In this study, we propose to use Reinforcement Learning to learn an end-to-end policy, mapping directly from tactile sensor readings to actions. Specifically, we use Dreamer-v3 on a challenging, partially observable robotic insertion task with a Franka Research 3, both in simulation and on a real system. For the real setup, we built a robotic platform capable of resetting itself fully autonomously, allowing for extensive training runs without human supervision. Our preliminary results indicate that Dreamer is capable of utilizing tactile inputs to solve robotic manipulation tasks in simulation and reality. Furthermore, we find that providing the robot with tactile feedback generally improves task performance, though, in our setup, we do not yet include other sensing modalities. In the future, we plan to utilize our platform to evaluate a wide range of other Reinforcement Learning algorithms on tactile tasks.