Distributed Reinforcement Learning for Cooperative Multi-Robot Object Manipulation

TL;DR

This paper introduces two distributed reinforcement learning methods for cooperative multi-robot object manipulation, demonstrating their effectiveness in simulation and potential scalability to larger multi-agent systems.

Contribution

It proposes two novel distributed multi-agent RL algorithms, DA-RL and GT-RL, tailored for cooperative multi-robot tasks, with validation in simulation.

Findings

Both methods successfully enable cooperative manipulation in simulation.

The approaches are scalable and applicable to larger multi-agent systems.

The methods outperform baseline approaches in cooperative tasks.

Abstract

We consider solving a cooperative multi-robot object manipulation task using reinforcement learning (RL). We propose two distributed multi-agent RL approaches: distributed approximate RL (DA-RL), where each agent applies Q-learning with individual reward functions; and game-theoretic RL (GT-RL), where the agents update their Q-values based on the Nash equilibrium of a bimatrix Q-value game. We validate the proposed approaches in the setting of cooperative object manipulation with two simulated robot arms. Although we focus on a small system of two agents in this paper, both DA-RL and GT-RL apply to general multi-agent systems, and are expected to scale well to large systems.