The Power of Communication in a Distributed Multi-Agent System

TL;DR

This paper introduces a distributed multi-agent reinforcement learning framework with communication capabilities, leveraging Dec-POMDPs and GNNs to improve efficiency and scalability in real-world applications like offshore wind farms.

Contribution

It presents a novel decentralized multi-agent learning mechanism with communication, combining Dec-POMDPs and GNNs, to enhance performance and reduce training time in complex environments.

Findings

Multi-agent collaboration reduces training time.

Multi-agent system achieves higher cumulative rewards.

System scales effectively to larger scenarios.

Abstract

Single-Agent (SA) Reinforcement Learning systems have shown outstanding re-sults on non-stationary problems. However, Multi-Agent Reinforcement Learning(MARL) can surpass SA systems generally and when scaling. Furthermore, MAsystems can be super-powered by collaboration, which can happen through ob-serving others, or a communication system used to share information betweencollaborators. Here, we developed a distributed MA learning mechanism withthe ability to communicate based on decentralised partially observable Markovdecision processes (Dec-POMDPs) and Graph Neural Networks (GNNs). Minimis-ing the time and energy consumed by training Machine Learning models whileimproving performance can be achieved by collaborative MA mechanisms. Wedemonstrate this in a real-world scenario, an offshore wind farm, including a set ofdistributed wind turbines, where the objective is to maximise collective efficiency.Compared to a SA system, MA collaboration has shown significantly reducedtraining time and higher cumulative rewards in unseen and scaled scenarios.