Reinforcement Learning with Time-dependent Goals for Robotic Musicians

TL;DR

This paper introduces time-dependent goals in reinforcement learning to enable robotic musicians to learn and perform musical sequences, specifically training a robot to play the theremin both in simulation and real-world settings.

Contribution

It presents a novel temporal extension to goal-conditioned reinforcement learning tailored for sequential tasks like music performance.

Findings

Successful transfer of the learned policy from simulation to real robot

Robotic thereminist can perform melodies with temporal coherence

Demonstrates feasibility of RL for complex sequential robotic tasks

Abstract

Reinforcement learning is a promising method to accomplish robotic control tasks. The task of playing musical instruments is, however, largely unexplored because it involves the challenge of achieving sequential goals - melodies - that have a temporal dimension. In this paper, we address robotic musicianship by introducing a temporal extension to goal-conditioned reinforcement learning: Time-dependent goals. We demonstrate that these can be used to train a robotic musician to play the theremin instrument. We train the robotic agent in simulation and transfer the acquired policy to a real-world robotic thereminist. Supplemental video: https://youtu.be/jvC9mPzdQN4