Centrally Coordinated Multi-Agent Reinforcement Learning for Power Grid Topology Control

TL;DR

This paper introduces a centrally coordinated multi-agent reinforcement learning framework for power grid topology control, demonstrating improved efficiency and performance over existing methods in complex, renewable-rich power systems.

Contribution

It proposes a novel CCMA architecture that effectively decomposes decision-making in power grid control, outperforming baseline approaches in various experimental settings.

Findings

Higher sample efficiency than baselines

Superior final performance in experiments

Potential for real-world power grid applications

Abstract

Power grid operation is becoming more complex due to the increase in generation of renewable energy. The recent series of Learning To Run a Power Network (L2RPN) competitions have encouraged the use of artificial agents to assist human dispatchers in operating power grids. However, the combinatorial nature of the action space poses a challenge to both conventional optimizers and learned controllers. Action space factorization, which breaks down decision-making into smaller sub-tasks, is one approach to tackle the curse of dimensionality. In this study, we propose a centrally coordinated multi-agent (CCMA) architecture for action space factorization. In this approach, regional agents propose actions and subsequently a coordinating agent selects the final action. We investigate several implementations of the CCMA architecture, and benchmark in different experimental settings against various L2RPN baseline approaches. The CCMA architecture exhibits higher sample efficiency and superior final performance than the baseline approaches. The results suggest high potential of the CCMA approach for further application in higher-dimensional L2RPN as well as real-world power grid settings.