Spatio-Temporal Graph Convolutional Neural Networks for Physics-Aware Grid Learning Algorithms

TL;DR

This paper introduces a novel spatio-temporal graph convolutional neural network framework combined with deep reinforcement learning for physics-aware Volt-VAR control in unbalanced distribution systems, enhancing stability and voltage regulation.

Contribution

It develops a new graph-based deep reinforcement learning approach with a physics-informed graph shift operator for improved Volt-VAR control in power systems.

Findings

Effective voltage stabilization in 123-bus systems

Superior performance over traditional methods

Robustness with partial system observations

Abstract

This paper proposes a model-free Volt-VAR control (VVC) algorithm via the spatio-temporal graph ConvNet-based deep reinforcement learning (STGCN-DRL) framework, whose goal is to control smart inverters in an unbalanced distribution system. We first identify the graph shift operator (GSO) based on the power flow equations. Then, we develop a spatio-temporal graph ConvNet (STGCN), testing both recurrent graph ConvNets (RGCN) and convolutional graph ConvNets (CGCN) architectures, aimed at capturing the spatiotemporal correlation of voltage phasors. The STGCN layer performs the feature extraction task for the policy function and the value function of the reinforcement learning architecture, and then we utilize the proximal policy optimization (PPO) to search the action spaces for an optimum policy function and to approximate an optimum value function. We further utilize the low-pass property of voltage graph signal to introduce an GCN architecture for the the policy whose input is a decimated state vector, i.e. a partial observation. Case studies on the unbalanced 123-bus systems validate the excellent performance of the proposed method in mitigating instabilities and maintaining nodal voltage profiles within a desirable range.