Reinforcement Learning for Resilient Power Grids

TL;DR

This paper enhances power grid resilience using reinforcement learning by developing a new simulator, optimizing action spaces, and applying low-rank neural network regularization to improve RL agent performance during large-scale blackouts.

Contribution

It introduces an updated power grid simulator based on Grid2Op, explores the impact of reduced action spaces, and applies low-rank neural network regularization to boost RL control effectiveness.

Findings

Reduced action spaces improve training efficiency.

Low-rank regularization enhances RL agent performance.

The new simulator supports large-scale blackout scenarios.

Abstract

Traditional power grid systems have become obsolete under more frequent and extreme natural disasters. Reinforcement learning (RL) has been a promising solution for resilience given its successful history of power grid control. However, most power grid simulators and RL interfaces do not support simulation of power grid under large-scale blackouts or when the network is divided into sub-networks. In this study, we proposed an updated power grid simulator built on Grid2Op, an existing simulator and RL interface, and experimented on limiting the action and observation spaces of Grid2Op. By testing with DDQN and SliceRDQN algorithms, we found that reduced action spaces significantly improve training performance and efficiency. In addition, we investigated a low-rank neural network regularization method for deep Q-learning, one of the most widely used RL algorithms, in this power grid control scenario. As a result, the experiment demonstrated that in the power grid simulation environment, adopting this method will significantly increase the performance of RL agents.