VolleyBots: A Testbed for Multi-Drone Volleyball Game Combining Motion Control and Strategic Play

TL;DR

VolleyBots introduces a multi-drone volleyball testbed that combines motion control and strategic play, enabling research on embodied intelligence with complex multi-agent interactions and demonstrating promising simulation-to-real transfer.

Contribution

The paper presents a novel drone-based volleyball platform integrating cooperative and competitive gameplay with a hierarchical policy approach for complex multi-agent tasks.

Findings

On-policy RL outperforms off-policy in simple tasks

Hierarchical policy achieves 69.5% win rate in 3v3 matches

Zero-shot sim-to-real transfer demonstrated on real drones

Abstract

Robot sports, characterized by well-defined objectives, explicit rules, and dynamic interactions, present ideal scenarios for demonstrating embodied intelligence. In this paper, we present VolleyBots, a novel robot sports testbed where multiple drones cooperate and compete in the sport of volleyball under physical dynamics. VolleyBots integrates three features within a unified platform: competitive and cooperative gameplay, turn-based interaction structure, and agile 3D maneuvering. These intertwined features yield a complex problem combining motion control and strategic play, with no available expert demonstrations. We provide a comprehensive suite of tasks ranging from single-drone drills to multi-drone cooperative and competitive tasks, accompanied by baseline evaluations of representative reinforcement learning (RL), multi-agent reinforcement learning (MARL) and game-theoretic algorithms. Simulation results show that on-policy RL methods outperform off-policy methods in single-agent tasks, but both approaches struggle in complex tasks that combine motion control and strategic play. We additionally design a hierarchical policy which achieves 69.5% win rate against the strongest baseline in the 3 vs 3 task, demonstrating its potential for tackling the complex interplay between low-level control and high-level strategy. To highlight VolleyBots' sim-to-real potential, we further demonstrate the zero-shot deployment of a policy trained entirely in simulation on real-world drones.