Policy Gradient-Based EMT-in-the-Loop Learning to Mitigate Sub-Synchronous Control Interactions

TL;DR

This paper presents a reinforcement learning approach using policy gradients within an EMT-in-the-loop simulation to adaptively tune control gains and mitigate sub-synchronous oscillations caused by control interactions in power grids.

Contribution

It introduces a novel RL-based framework with signal processing for adaptive gain tuning to suppress sub-synchronous oscillations in power systems.

Findings

RL-trained policy effectively suppresses oscillations under varying grid conditions.

Deep policy gradient methods adaptively tune control gains in real-time.

The approach demonstrates improved stability compared to traditional fixed-gain methods.

Abstract

This paper explores the development of learning-based tunable control gains using EMT-in-the-loop simulation framework (e.g., PSCAD interfaced with Python-based learning modules) to address critical sub-synchronous oscillations. Since sub-synchronous control interactions (SSCI) arise from the mis-tuning of control gains under specific grid configurations, effective mitigation strategies require adaptive re-tuning of these gains. Such adaptiveness can be achieved by employing a closed-loop, learning-based framework that considers the grid conditions responsible for such sub-synchronous oscillations. This paper addresses this need by adopting methodologies inspired by Markov decision process (MDP) based reinforcement learning (RL), with a particular emphasis on simpler deep policy gradient methods with additional SSCI-specific signal processing modules such as down-sampling, bandpass filtering, and oscillation energy dependent reward computations. Our experimentation in a real-world event setting demonstrates that the deep policy gradient based trained policy can adaptively compute gain settings in response to varying grid conditions and optimally suppress control interaction-induced oscillations.