A Review of Cooperative Multi-Agent Deep Reinforcement Learning

TL;DR

This review comprehensively covers recent cooperative multi-agent deep reinforcement learning approaches, challenges, emerging research areas, real-world applications, and future research directions, providing a broad overview of the field.

Contribution

It systematically categorizes and analyzes five key approaches in cooperative MARL, connecting existing methods and highlighting new research trends and applications.

Findings

Identified five main approaches to cooperative MARL.

Reviewed recent applications of MARL in real-world scenarios.

Provided a list of environments for MARL research.

Abstract

Deep Reinforcement Learning has made significant progress in multi-agent systems in recent years. In this review article, we have focused on presenting recent approaches on Multi-Agent Reinforcement Learning (MARL) algorithms. In particular, we have focused on five common approaches on modeling and solving cooperative multi-agent reinforcement learning problems: (I) independent learners, (II) fully observable critic, (III) value function factorization, (IV) consensus, and (IV) learn to communicate. First, we elaborate on each of these methods, possible challenges, and how these challenges were mitigated in the relevant papers. If applicable, we further make a connection among different papers in each category. Next, we cover some new emerging research areas in MARL along with the relevant recent papers. Due to the recent success of MARL in real-world applications, we assign a section to provide a review of these applications and corresponding articles. Also, a list of available environments for MARL research is provided in this survey. Finally, the paper is concluded with proposals on the possible research directions.