High Acceleration Reinforcement Learning for Real-World Juggling with Binary Rewards

TL;DR

This paper presents a novel reinforcement learning system enabling a robot to learn juggling with high acceleration using only 56 minutes of experience and binary rewards, emphasizing safety and sample efficiency.

Contribution

The work introduces a policy design and optimization approach tailored for high-acceleration tasks with binary rewards, demonstrating effective real-world learning for dynamic juggling.

Findings

Robot learned to juggle in 56 minutes

Achieved continuous juggling for 33 minutes

Demonstrated safety and efficiency in real-world learning

Abstract

Robots that can learn in the physical world will be important to en-able robots to escape their stiff and pre-programmed movements. For dynamic high-acceleration tasks, such as juggling, learning in the real-world is particularly challenging as one must push the limits of the robot and its actuation without harming the system, amplifying the necessity of sample efficiency and safety for robot learning algorithms. In contrast to prior work which mainly focuses on the learning algorithm, we propose a learning system, that directly incorporates these requirements in the design of the policy representation, initialization, and optimization. We demonstrate that this system enables the high-speed Barrett WAM manipulator to learn juggling two balls from 56 minutes of experience with a binary reward signal. The final policy juggles continuously for up to 33 minutes or about 4500 repeated catches. The videos documenting the learning process and the evaluation can be found at https://sites.google.com/view/jugglingbot