Optimal Stroke Learning with Policy Gradient Approach for Robotic Table Tennis

TL;DR

This paper introduces a policy gradient method based on TD3 for robotic table tennis, achieving high success rates in real scenarios with efficient training and domain transfer from simulation.

Contribution

A novel policy gradient approach with TD3 backbone for learning robotic table tennis strokes, enabling effective simulation-to-reality transfer with minimal training.

Findings

Achieved 98% success rate in real scenarios

Reduced training time to approximately 1.5 hours

Outperformed existing RL methods in simulation

Abstract

Learning to play table tennis is a challenging task for robots, as a wide variety of strokes required. Recent advances have shown that deep Reinforcement Learning (RL) is able to successfully learn the optimal actions in a simulated environment. However, the applicability of RL in real scenarios remains limited due to the high exploration effort. In this work, we propose a realistic simulation environment in which multiple models are built for the dynamics of the ball and the kinematics of the robot. Instead of training an end-to-end RL model, a novel policy gradient approach with TD3 backbone is proposed to learn the racket strokes based on the predicted state of the ball at the hitting time. In the experiments, we show that the proposed approach significantly outperforms the existing RL methods in simulation. Furthermore, to cross the domain from simulation to reality, we adopt an efficient retraining method and test it in three real scenarios. The resulting success rate is 98% and the distance error is around 24.9 cm. The total training time is about 1.5 hours.