Learning coordinated badminton skills for legged manipulators

TL;DR

This paper presents a reinforcement learning approach enabling legged robots to play badminton by coordinating perception, locomotion, and arm control, validated through extensive real-world experiments.

Contribution

It introduces a unified visuomotor control policy with active perception and system identification for dynamic sports tasks in legged manipulators.

Findings

Robots can predict shuttlecock trajectories accurately.

Robots navigate and position effectively in complex environments.

Robots execute precise badminton strikes against humans.

Abstract

Coordinating the motion between lower and upper limbs and aligning limb control with perception are substantial challenges in robotics, particularly in dynamic environments. To this end, we introduce an approach for enabling legged mobile manipulators to play badminton, a task that requires precise coordination of perception, locomotion, and arm swinging. We propose a unified reinforcement learning-based control policy for whole-body visuomotor skills involving all degrees of freedom to achieve effective shuttlecock tracking and striking. This policy is informed by a perception noise model that utilizes real-world camera data, allowing for consistent perception error levels between simulation and deployment and encouraging learned active perception behaviors. Our method includes a shuttlecock prediction model, constrained reinforcement learning for robust motion control, and integrated system identification techniques to enhance deployment readiness. Extensive experimental results in a variety of environments validate the robot's capability to predict shuttlecock trajectories, navigate the service area effectively, and execute precise strikes against human players, demonstrating the feasibility of using legged mobile manipulators in complex and dynamic sports scenarios.