Deep Reinforcement Learning for Tactile Robotics: Learning to Type on a Braille Keyboard

TL;DR

This paper introduces a new tactile reinforcement learning environment for training robots to type on a braille keyboard, demonstrating successful simulation and real-world learning with tactile images, and providing resources for future research.

Contribution

It presents the first successful training of deep RL agents in the real world using only tactile image observations, along with a new environment, tasks, and open-source resources.

Findings

All tasks learned successfully in simulation.

Three out of four tasks learned on the real robot.

Continuous alphabet task remains impractical due to sample inefficiency.

Abstract

Artificial touch would seem well-suited for Reinforcement Learning (RL), since both paradigms rely on interaction with an environment. Here we propose a new environment and set of tasks to encourage development of tactile reinforcement learning: learning to type on a braille keyboard. Four tasks are proposed, progressing in difficulty from arrow to alphabet keys and from discrete to continuous actions. A simulated counterpart is also constructed by sampling tactile data from the physical environment. Using state-of-the-art deep RL algorithms, we show that all of these tasks can be successfully learnt in simulation, and 3 out of 4 tasks can be learned on the real robot. A lack of sample efficiency currently makes the continuous alphabet task impractical on the robot. To the best of our knowledge, this work presents the first demonstration of successfully training deep RL agents in the real world using observations that exclusively consist of tactile images. To aid future research utilising this environment, the code for this project has been released along with designs of the braille keycaps for 3D printing and a guide for recreating the experiments. A brief video summary is also available at https://youtu.be/eNylCA2uE_E.