A Reinforcement Learning Approach for Fast Frequency Control in Low-Inertia Power Systems

TL;DR

This paper introduces a reinforcement learning-based control method for fast frequency regulation in low-inertia power systems, leveraging grid-forming converters to enhance stability amid increasing renewable integration.

Contribution

It presents a novel RL-driven control scheme for VSCs that improves frequency response in low-inertia grids, validated through detailed simulations.

Findings

RL controller effectively maintains frequency stability.

Fast response of VSCs enhances system resilience.

Potential for real-time implementation in future grids.

Abstract

The electric grid is undergoing a major transition from fossil fuel-based power generation to renewable energy sources, typically interfaced to the grid via power electronics. The future power systems are thus expected to face increased control complexity and challenges pertaining to frequency stability due to lower levels of inertia and damping. As a result, the frequency control and development of novel ancillary services is becoming imperative. This paper proposes a data-driven control scheme, based on Reinforcement Learning (RL), for grid-forming Voltage Source Converters (VSCs), with the goal of exploiting their fast response capabilities to provide fast frequency control to the system. A centralized RL-based controller collects generator frequencies and adjusts the VSC power output, in response to a disturbance, to prevent frequency threshold violations. The proposed control scheme is analyzed and its performance evaluated through detailed time-domain simulations of the IEEE 14-bus test system.