Modular Reinforcement Learning For Cooperative Swarms

TL;DR

This paper introduces a modular approach to multi-robot reinforcement learning in swarms, enabling efficient learning of interactions for cooperative foraging tasks.

Contribution

It proposes a decomposed state representation method that reduces memory demands and improves learning efficiency in robot swarms.

Findings

The modular approach outperforms traditional methods in simulated foraging tasks.

Decomposed state features lead to better scalability in swarm reinforcement learning.

Experimental results show improved cooperation among robots using the proposed method.

Abstract

A cooperative robot swarm is a collective of computationally-limited robots that share a common goal. Each robot can only interact with a small subset of its peers, without knowing how this affects the collective utility. Recent advances in distributed multi-agent reinforcement learning have demonstrated that it is possible for robots to learn how to interact effectively with others, in a manner that is aligned with the common goal, despite each robot learning independently of others. However, this requires each robot to represent a potentially combinatorial number of interaction states, challenging the memory capabilities of the robots. This paper proposes an alternative approach for representing spatial interaction states for multi-robot reinforcement learning in swarms. A modular (decomposed) representation is used, where each feature of the state is handled by a separate learning procedure, and the results aggregated. We demonstrate the efficacy of the approach in numerous experiments with simulated robot swarms carrying out foraging.