Distributed Reinforcement Learning using Local Smart Meter Data for Voltage Regulation in Distribution Networks

TL;DR

This paper introduces a distributed reinforcement learning approach for voltage regulation in distribution networks that leverages local smart meter data and a novel correction strategy to improve accuracy and coordination, reducing reliance on centralized computations.

Contribution

The paper presents a novel distributed RL algorithm with local voltage estimation, a correction strategy combining neural networks and piecewise functions, and an online coordination method for improved voltage regulation.

Findings

Effective voltage regulation demonstrated in case studies

Reduced computational overhead and privacy concerns

Improved accuracy with the correction strategy

Abstract

Centralised reinforcement learning (RL) for voltage magnitude regulation in distribution networks typically involves numerous agent-environment interactions and power flow (PF) calculations, inducing computational overhead and privacy concerns over shared data. Thus, we propose a distributed RL algorithm to regulate voltage magnitude. First, a dynamic Thevenin equivalent model is integrated within smart meters (SM), enabling local voltage magnitude estimation using local SM data for RL agent training, and mitigating the dependency of synchronised data collection and centralised PF calculations. To mitigate estimation errors induced by Thevenin model inaccuracies, a voltage magnitude correction strategy that combines piecewise functions and neural networks is introduced. The piecewise function corrects the large errors of estimated voltage magnitude, while a neural network mimics the grid's sensitivity to control actions, improving action adjustment precision. Second, a coordination strategy is proposed to refine local RL agent actions online, preventing voltage magnitude violations induced by excessive actions from multiple independently trained agents. Case studies on energy storage systems validate the feasibility and effectiveness of the proposed approach, demonstrating its potential to improve voltage regulation in distribution networks.