Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Control

TL;DR

This paper introduces a safety-constrained multi-agent reinforcement learning algorithm for active voltage control in power networks, ensuring safety constraints while optimizing voltage regulation in real-world scenarios.

Contribution

It formalizes the voltage control problem as a constrained Markov game and develops a novel primal-dual RL method with double safety estimation for multi-agent settings.

Findings

The proposed method guarantees safety constraints in voltage control.

Experimental results outperform state-of-the-art MARL approaches.

Effective in real-world scale power distribution scenarios.

Abstract

Active voltage control presents a promising avenue for relieving power congestion and enhancing voltage quality, taking advantage of the distributed controllable generators in the power network, such as roof-top photovoltaics. While Multi-Agent Reinforcement Learning (MARL) has emerged as a compelling approach to address this challenge, existing MARL approaches tend to overlook the constrained optimization nature of this problem, failing in guaranteeing safety constraints. In this paper, we formalize the active voltage control problem as a constrained Markov game and propose a safety-constrained MARL algorithm. We expand the primal-dual optimization RL method to multi-agent settings, and augment it with a novel approach of double safety estimation to learn the policy and to update the Lagrange-multiplier. In addition, we proposed different cost functions and investigated their influences on the behavior of our constrained MARL method. We evaluate our approach in the power distribution network simulation environment with real-world scale scenarios. Experimental results demonstrate the effectiveness of the proposed method compared with the state-of-the-art MARL methods. This paper is published at \url{https://www.ijcai.org/Proceedings/2024/}.