Sample-efficient Reinforcement Learning in Robotic Table Tennis

TL;DR

This paper introduces a sample-efficient reinforcement learning algorithm tailored for robotic table tennis, achieving effective learning within 200 episodes by leveraging a one-step environment and actor-critic methods.

Contribution

The paper presents a novel, sample-efficient RL approach for robotic table tennis that operates with minimal trials by embedding the method into the robot system and using a one-step environment.

Findings

Achieves competitive performance in simulation and real robot scenarios.

Learns accurate return strategies without pre-training in under 200 episodes.

Demonstrates effectiveness in high-dimensional, continuous state spaces.

Abstract

Reinforcement learning (RL) has achieved some impressive recent successes in various computer games and simulations. Most of these successes are based on having large numbers of episodes from which the agent can learn. In typical robotic applications, however, the number of feasible attempts is very limited. In this paper we present a sample-efficient RL algorithm applied to the example of a table tennis robot. In table tennis every stroke is different, with varying placement, speed and spin. An accurate return therefore has to be found depending on a high-dimensional continuous state space. To make learning in few trials possible the method is embedded into our robot system. In this way we can use a one-step environment. The state space depends on the ball at hitting time (position, velocity, spin) and the action is the racket state (orientation, velocity) at hitting. An actor-critic based deterministic policy gradient algorithm was developed for accelerated learning. Our approach performs competitively both in a simulation and on the real robot in a number of challenging scenarios. Accurate results are obtained without pre-training in under $200$ episodes of training. The video presenting our experiments is available at https://youtu.be/uRAtdoL6Wpw.