Optimization for Reinforcement Learning: From Single Agent to Cooperative Agents

TL;DR

This paper reviews recent advances in multi-agent reinforcement learning, focusing on decentralized algorithms for large-scale control and communication, highlighting the evolution from single-agent to cooperative multi-agent systems and discussing future challenges.

Contribution

It provides a comprehensive overview of multi-agent reinforcement learning algorithms, emphasizing decentralized methods and their development from single-agent approaches.

Findings

Highlights the transition from single-agent to multi-agent RL algorithms.

Emphasizes decentralized control and communication protocols.

Identifies future research directions and challenges.

Abstract

This article reviews recent advances in multi-agent reinforcement learning algorithms for large-scale control systems and communication networks, which learn to communicate and cooperate. We provide an overview of this emerging field, with an emphasis on the decentralized setting under different coordination protocols. We highlight the evolution of reinforcement learning algorithms from single-agent to multi-agent systems, from a distributed optimization perspective, and conclude with future directions and challenges, in the hope to catalyze the growing synergy among distributed optimization, signal processing, and reinforcement learning communities.