Fully Decentralized Cooperative Multi-Agent Reinforcement Learning: A Survey

TL;DR

This survey reviews fully decentralized cooperative multi-agent reinforcement learning methods, highlighting challenges, existing approaches, and future research directions for training agents without centralized information.

Contribution

It systematically categorizes and discusses fully decentralized methods in cooperative multi-agent reinforcement learning, an area with limited prior comprehensive reviews.

Findings

Identifies key challenges in fully decentralized settings.

Summarizes existing algorithms for shared and individual reward maximization.

Discusses open questions and future research directions.

Abstract

Cooperative multi-agent reinforcement learning is a powerful tool to solve many real-world cooperative tasks, but restrictions of real-world applications may require training the agents in a fully decentralized manner. Due to the lack of information about other agents, it is challenging to derive algorithms that can converge to the optimal joint policy in a fully decentralized setting. Thus, this research area has not been thoroughly studied. In this paper, we seek to systematically review the fully decentralized methods in two settings: maximizing a shared reward of all agents and maximizing the sum of individual rewards of all agents, and discuss open questions and future research directions.