Barrier Function-based Safe Reinforcement Learning for Emergency Control of Power Systems

TL;DR

This paper presents a novel safe reinforcement learning approach for emergency load shedding in power systems, using barrier functions to ensure voltage safety and improve control effectiveness during faults.

Contribution

The paper introduces a barrier function-based safe RL method that guarantees system safety during emergency control in power grids, a novel approach in this domain.

Findings

Effective voltage recovery demonstrated on IEEE 39-bus system

Method adapts to unseen faults during testing

Enhanced safety guarantees over standard RL methods

Abstract

Under voltage load shedding has been considered as a standard and effective measure to recover the voltage stability of the electric power grid under emergency and severe conditions. However, this scheme usually trips a massive amount of load which can be unnecessary and harmful to customers. Recently, deep reinforcement learning (RL) has been regarded and adopted as a promising approach that can significantly reduce the amount of load shedding. However, like most existing machine learning (ML)-based control techniques, RL control usually cannot guarantee the safety of the systems under control. In this paper, we introduce a novel safe RL method for emergency load shedding of power systems, that can enhance the safe voltage recovery of the electric power grid after experiencing faults. Unlike the standard RL method, the safe RL method has a reward function consisting of a Barrier function that goes to minus infinity when the system state goes to the safety bounds. Consequently, the optimal control policy, that maximizes the reward function, can render the power system to avoid the safety bounds. This method is general and can be applied to other safety-critical control problems. Numerical simulations on the 39-bus IEEE benchmark is performed to demonstrate the effectiveness of the proposed safe RL emergency control, as well as its adaptive capability to faults not seen in the training.