Coordinated Frequency Control through Safe Reinforcement Learning

TL;DR

This paper introduces a safe reinforcement learning-based framework for real-time, coordinated frequency control in power grids, addressing the limitations of traditional methods amid increasing renewable integration.

Contribution

It presents a novel model-free, multi-objective frequency control approach using safe reinforcement learning, enabling rapid, coordinated decisions in power grid management.

Findings

Effective in achieving sub-second control decisions

Demonstrated success in East China Power Grid simulations

Outperforms traditional frequency control methods

Abstract

With widespread deployment of renewables, the electric power grids are experiencing increasing dynamics and uncertainties, with its secure operation being threatened. Existing frequency control schemes based on day-ahead offline analysis and minute-level online sensitivity calculations are difficult to adapt to rapidly changing system states. In particular, they are unable to facilitate coordinated control of system frequency and power flows. A refined approach and tools are urgently needed to assist system operators to make timely decisions. This paper proposes a novel model-free coordinated frequency control framework based on safe reinforcement learning, with multiple control objectives considered. The load frequency control problem is modeled as a constrained Markov decision process, which can be solved by an AI agent continuously interacting with the grid to achieve sub-second decision making. Extensive numerical experiments conducted at East China Power Grid demonstrate the effectiveness and promise of the proposed method.