Universal Policies to Learn Them All

TL;DR

This paper introduces a universal multi-agent reinforcement learning algorithm that generalizes across multiple scenarios, demonstrated in a new urban security environment where traditional methods fail.

Contribution

The paper presents a novel multi-agent RL algorithm inspired by universal value function approximators, enabling generalization over states and scenarios.

Findings

The proposed method outperforms state-of-the-art algorithms in multi-scenario generalization.

Traditional algorithms fail to generalize across multiple scenarios.

The new environment effectively tests multi-agent RL generalization capabilities.

Abstract

We explore a collaborative and cooperative multi-agent reinforcement learning setting where a team of reinforcement learning agents attempt to solve a single cooperative task in a multi-scenario setting. We propose a novel multi-agent reinforcement learning algorithm inspired by universal value function approximators that not only generalizes over state space but also over a set of different scenarios. Additionally, to prove our claim, we are introducing a challenging 2D multi-agent urban security environment where the learning agents are trying to protect a person from nearby bystanders in a variety of scenarios. Our study shows that state-of-the-art multi-agent reinforcement learning algorithms fail to generalize a single task over multiple scenarios while our proposed solution works equally well as scenario-dependent policies.