A Survey of Multi Agent Reinforcement Learning: Federated Learning and Cooperative and Noncooperative Decentralized Regimes

TL;DR

This survey comprehensively reviews multi-agent reinforcement learning across federated, decentralized, and noncooperative regimes, highlighting recent developments, theoretical guarantees, and performance limitations.

Contribution

It provides a structured comparison of three interaction topologies in multi-agent RL, summarizing recent advances, formal formulations, and theoretical insights.

Findings

Identifies structural similarities and differences among the three regimes.

Summarizes recent theoretical guarantees in multi-agent RL.

Highlights limitations in current numerical performance evaluations.

Abstract

The increasing interest in research and innovation towards the development of autonomous agents presents a number of complex yet important scenarios of multiple AI Agents interacting with each other in an environment. The particular setting can be understood as exhibiting three possibly topologies of interaction - centrally coordinated cooperation, ad-hoc interaction and cooperation, and settings with noncooperative incentive structures. This article presents a comprehensive survey of all three domains, defined under the formalism of Federal Reinforcement Learning (RL), Decentralized RL, and Noncooperative RL, respectively. Highlighting the structural similarities and distinctions, we review the state of the art in these subjects, primarily explored and developed only recently in the literature. We include the formulations as well as known theoretical guarantees and highlights and limitations of numerical performance.