R-MADDPG for Partially Observable Environments and Limited Communication

TL;DR

This paper introduces R-MADDPG, a deep recurrent multiagent reinforcement learning framework designed to improve coordination in partially observable environments with limited communication, demonstrating effective learning of communication patterns and resource sharing.

Contribution

The paper presents a novel R-MADDPG framework that incorporates recurrence to handle partial observability and limited communication in multiagent systems, advancing coordination capabilities.

Findings

Learns time dependencies for sharing observations

Develops different communication patterns among agents

Handles resource limitations effectively

Abstract

There are several real-world tasks that would benefit from applying multiagent reinforcement learning (MARL) algorithms, including the coordination among self-driving cars. The real world has challenging conditions for multiagent learning systems, such as its partial observable and nonstationary nature. Moreover, if agents must share a limited resource (e.g. network bandwidth) they must all learn how to coordinate resource use. This paper introduces a deep recurrent multiagent actor-critic framework (R-MADDPG) for handling multiagent coordination under partial observable set-tings and limited communication. We investigate recurrency effects on performance and communication use of a team of agents. We demonstrate that the resulting framework learns time dependencies for sharing missing observations, handling resource limitations, and developing different communication patterns among agents.