Toward Real-World Cooperative and Competitive Soccer with Quadrupedal Robot Teams

TL;DR

This paper introduces a hierarchical multi-agent reinforcement learning framework enabling autonomous quadruped robot teams to perform complex cooperative and competitive soccer tasks, demonstrating real-world deployment and sophisticated team behaviors.

Contribution

It presents a novel decentralized MARL approach with hierarchical skills training and strategic planning for real-world quadruped robot soccer.

Findings

Agents achieve coordinated passing and interception behaviors

The system performs well in both indoor and outdoor environments

Decentralized policies enable autonomous real-world robot soccer matches

Abstract

Achieving coordinated teamwork among legged robots requires both fine-grained locomotion control and long-horizon strategic decision-making. Robot soccer offers a compelling testbed for this challenge, combining dynamic, competitive, and multi-agent interactions. In this work, we present a hierarchical multi-agent reinforcement learning (MARL) framework that enables fully autonomous and decentralized quadruped robot soccer. First, a set of highly dynamic low-level skills is trained for legged locomotion and ball manipulation, such as walking, dribbling, and kicking. On top of these, a high-level strategic planning policy is trained with Multi-Agent Proximal Policy Optimization (MAPPO) via Fictitious Self-Play (FSP). This learning framework allows agents to adapt to diverse opponent strategies and gives rise to sophisticated team behaviors, including coordinated passing, interception, and dynamic role allocation. With an extensive ablation study, the proposed learning method shows significant advantages in the cooperative and competitive multi-agent soccer game. We deploy the learned policies to real quadruped robots relying solely on onboard proprioception and decentralized localization, with the resulting system supporting autonomous robot-robot and robot-human soccer matches on indoor and outdoor soccer courts.