Diff-Muscle: Efficient Learning for Musculoskeletal Robotic Table Tennis

TL;DR

Diff-Muscle introduces a novel control algorithm for musculoskeletal robots that leverages differential flatness and hierarchical reinforcement learning to enable dexterous, efficient, and successful robotic table tennis play, outperforming existing methods.

Contribution

The paper presents a new control framework combining differential flatness and hierarchical reinforcement learning for musculoskeletal robots, enabling efficient learning and complex multi-segment coordination.

Findings

Outperforms state-of-the-art baselines in success rates.

Maintains minimal muscle activation during tasks.

Enables continuous rallies in dual-robot table tennis.

Abstract

Musculoskeletal robots provide superior advantages in flexibility and dexterity, positioning them as a promising frontier towards embodied intelligence. However, current research is largely confined to relative simple tasks, restricting the exploration of their full potential in multi-segment coordination. Furthermore, efficient learning remains a challenge, primarily due to the high-dimensional action space and inherent overactuated structures. To address these challenges, we propose Diff-Muscle, a musculoskeletal robot control algorithm that leverages differential flatness to reformulate policy learning from the redundant muscle-activation space into a significantly lower-dimensional joint space. Furthermore, we utilize the highly dynamic robotic table tennis task to evaluate our algorithm. Specifically, we propose a hierarchical reinforcement learning framework that integrates a Kinematics-based Muscle Actuation Controller (K-MAC) with high-level trajectory planning, enabling a musculoskeletal robot to perform dexterous and precise rallies. Experimental results demonstrate that Diff-Muscle significantly outperforms state-of-the-art baselines in success rates while maintaining minimal muscle activation. Notably, the proposed framework successfully enables the musculoskeletal robots to achieve continuous rallies in a challenging dual-robot setting.