Multi-Agent Reinforcement Learning: A Report on Challenges and Approaches

TL;DR

This paper reviews the unique challenges of multi-agent reinforcement learning in mixed environments and discusses recent advances like decentralized actor-critic methods based on decentralized POMDPs.

Contribution

It highlights the specific challenges in multi-agent RL and discusses recent approaches such as decentralized actor-critic algorithms based on decentralized POMDPs.

Findings

Decentralized actor-critic methods show promise in multi-agent RL.

Challenges include coordination and partial observability in multi-agent settings.

Recent advances address some limitations of single-agent RL in multi-agent environments.

Abstract

Reinforcement Learning (RL) is a learning paradigm concerned with learning to control a system so as to maximize an objective over the long term. This approach to learning has received immense interest in recent times and success manifests itself in the form of human-level performance on games like \textit{Go}. While RL is emerging as a practical component in real-life systems, most successes have been in Single Agent domains. This report will instead specifically focus on challenges that are unique to Multi-Agent Systems interacting in mixed cooperative and competitive environments. The report concludes with advances in the paradigm of training Multi-Agent Systems called \textit{Decentralized Actor, Centralized Critic}, based on an extension of MDPs called \textit{Decentralized Partially Observable MDP}s, which has seen a renewed interest lately.