Cooperative-Competitive Team Play of Real-World Craft Robots

TL;DR

This paper presents a comprehensive multi-robot system and RL techniques for efficient cooperative and competitive policy training, including a novel sim-to-real transfer method that improves real-world performance.

Contribution

It introduces a new robotic platform, RL training methods for multi-agent cooperation and competition, and a novel Out of Distribution State Initialization technique for better sim-to-real transfer.

Findings

OODSI improves Sim2Real transfer by 20%.

The system successfully performs in real-world multi-robot tasks.

RL techniques enable effective cooperative and competitive behaviors.

Abstract

Multi-agent deep Reinforcement Learning (RL) has made significant progress in developing intelligent game-playing agents in recent years. However, the efficient training of collective robots using multi-agent RL and the transfer of learned policies to real-world applications remain open research questions. In this work, we first develop a comprehensive robotic system, including simulation, distributed learning framework, and physical robot components. We then propose and evaluate reinforcement learning techniques designed for efficient training of cooperative and competitive policies on this platform. To address the challenges of multi-agent sim-to-real transfer, we introduce Out of Distribution State Initialization (OODSI) to mitigate the impact of the sim-to-real gap. In the experiments, OODSI improves the Sim2Real performance by 20%. We demonstrate the effectiveness of our approach through experiments with a multi-robot car competitive game and a cooperative task in real-world settings.