Consensus Multi-Agent Reinforcement Learning for Volt-VAR Control in Power Distribution Networks

TL;DR

This paper introduces a consensus multi-agent deep reinforcement learning approach for Volt-VAR control in power distribution networks, enhancing robustness and communication efficiency without relying on detailed network models.

Contribution

It proposes a novel multi-agent reinforcement learning algorithm with a consensus strategy for VVC, improving resilience and reducing communication needs.

Findings

Matches single-agent RL performance on IEEE test feeders

Demonstrates communication efficiency in multi-agent setting

Shows resilience against communication or controller failures

Abstract

Volt-VAR control (VVC) is a critical application in active distribution network management system to reduce network losses and improve voltage profile. To remove dependency on inaccurate and incomplete network models and enhance resiliency against communication or controller failure, we propose consensus multi-agent deep reinforcement learning algorithm to solve the VVC problem. The VVC problem is formulated as a networked multi-agent Markov decision process, which is solved using the maximum entropy reinforcement learning framework and a novel communication-efficient consensus strategy. The proposed algorithm allows individual agents to learn a group control policy using local rewards. Numerical studies on IEEE distribution test feeders show that our proposed algorithm matches the performance of single-agent reinforcement learning benchmark. In addition, the proposed algorithm is shown to be communication efficient and resilient.