Stabilizing Voltage in Power Distribution Networks via Multi-Agent Reinforcement Learning with Transformer

TL;DR

This paper introduces T-MAAC, a transformer-based multi-agent reinforcement learning framework, to improve voltage stabilization in power distribution networks with renewable energy integration, enhancing sample efficiency and control performance.

Contribution

It presents a novel transformer-based MARL framework with auxiliary-task training tailored for voltage control in power grids, addressing limitations of prior approaches.

Findings

T-MAAC outperforms existing MARL methods in voltage stabilization tasks.

The auxiliary-task training improves sample efficiency and learning stability.

Transformer representations adapt well to power network problems.

Abstract

The increased integration of renewable energy poses a slew of technical challenges for the operation of power distribution networks. Among them, voltage fluctuations caused by the instability of renewable energy are receiving increasing attention. Utilizing MARL algorithms to coordinate multiple control units in the grid, which is able to handle rapid changes of power systems, has been widely studied in active voltage control task recently. However, existing approaches based on MARL ignore the unique nature of the grid and achieve limited performance. In this paper, we introduce the transformer architecture to extract representations adapting to power network problems and propose a Transformer-based Multi-Agent Actor-Critic framework (T-MAAC) to stabilize voltage in power distribution networks. In addition, we adopt a novel auxiliary-task training process tailored to the voltage control task, which improves the sample efficiency and facilitating the representation learning of the transformer-based model. We couple T-MAAC with different multi-agent actor-critic algorithms, and the consistent improvements on the active voltage control task demonstrate the effectiveness of the proposed method.