Multi-Agent Reinforcement Learning for Active Voltage Control on Power Distribution Networks

TL;DR

This paper explores the application of multi-agent reinforcement learning to active voltage control in power distribution networks, addressing challenges and proposing a new environment for real-world deployment.

Contribution

It formulates the voltage control problem as a Dec-POMDP, creates an open-source environment, and analyzes challenges for MARL in power systems.

Findings

Formulated active voltage control as Dec-POMDP

Developed an open-source MARL environment for power networks

Identified interpretability as a key challenge for MARL in this domain

Abstract

This paper presents a problem in power networks that creates an exciting and yet challenging real-world scenario for application of multi-agent reinforcement learning (MARL). The emerging trend of decarbonisation is placing excessive stress on power distribution networks. Active voltage control is seen as a promising solution to relieve power congestion and improve voltage quality without extra hardware investment, taking advantage of the controllable apparatuses in the network, such as roof-top photovoltaics (PVs) and static var compensators (SVCs). These controllable apparatuses appear in a vast number and are distributed in a wide geographic area, making MARL a natural candidate. This paper formulates the active voltage control problem in the framework of Dec-POMDP and establishes an open-source environment. It aims to bridge the gap between the power community and the MARL community and be a drive force towards real-world applications of MARL algorithms. Finally, we analyse the special characteristics of the active voltage control problems that cause challenges (e.g. interpretability) for state-of-the-art MARL approaches, and summarise the potential directions.