A Survey and Critique of Multiagent Deep Reinforcement Learning

TL;DR

This paper provides a comprehensive overview and critique of multiagent deep reinforcement learning, highlighting key components, challenges, and future research directions to unify and advance the field.

Contribution

It offers a detailed survey of MDRL literature, revisits foundational components, and provides practical guidelines and critical insights for future research.

Findings

Highlights adaptation of RL components to multiagent settings

Identifies practical challenges like computational demands

Provides benchmarks and open research avenues

Abstract

Deep reinforcement learning (RL) has achieved outstanding results in recent years. This has led to a dramatic increase in the number of applications and methods. Recent works have explored learning beyond single-agent scenarios and have considered multiagent learning (MAL) scenarios. Initial results report successes in complex multiagent domains, although there are several challenges to be addressed. The primary goal of this article is to provide a clear overview of current multiagent deep reinforcement learning (MDRL) literature. Additionally, we complement the overview with a broader analysis: (i) we revisit previous key components, originally presented in MAL and RL, and highlight how they have been adapted to multiagent deep reinforcement learning settings. (ii) We provide general guidelines to new practitioners in the area: describing lessons learned from MDRL works, pointing to recent benchmarks, and outlining open avenues of research. (iii) We take a more critical tone raising practical challenges of MDRL (e.g., implementation and computational demands). We expect this article will help unify and motivate future research to take advantage of the abundant literature that exists (e.g., RL and MAL) in a joint effort to promote fruitful research in the multiagent community.