Decentralized Multi-Agent Reinforcement Learning with Networked Agents: Recent Advances

TL;DR

This paper reviews recent advances in decentralized multi-agent reinforcement learning where agents communicate over networks without central control, highlighting theoretical developments and practical applications.

Contribution

It synthesizes recent research on decentralized MARL with networked agents, emphasizing theoretical analysis and practical applications in various control systems.

Findings

Recent algorithms improve coordination without central control

Theoretical analysis supports convergence and stability

Applications span robotics, sensor networks, and smart grids

Abstract

Multi-agent reinforcement learning (MARL) has long been a significant and everlasting research topic in both machine learning and control. With the recent development of (single-agent) deep RL, there is a resurgence of interests in developing new MARL algorithms, especially those that are backed by theoretical analysis. In this paper, we review some recent advances a sub-area of this topic: decentralized MARL with networked agents. Specifically, multiple agents perform sequential decision-making in a common environment, without the coordination of any central controller. Instead, the agents are allowed to exchange information with their neighbors over a communication network. Such a setting finds broad applications in the control and operation of robots, unmanned vehicles, mobile sensor networks, and smart grid. This review is built upon several our research endeavors in this direction, together with some progresses made by other researchers along the line. We hope this review to inspire the devotion of more research efforts to this exciting yet challenging area.