Hierarchical Decision Making In Electricity Grid Management

TL;DR

This paper introduces a hierarchical reinforcement learning approach to improve decision making in electricity grid management, addressing the challenges of uncertainty and multiple decision time scales.

Contribution

It presents a novel hierarchical RL model and algorithm tailored for power grid reliability, with an abstraction level for real-time decision making.

Findings

Outperforms existing heuristics in reliability management

Effectively handles stochastic renewable energy and demand fluctuations

Demonstrates improved decision efficiency in complex grid scenarios

Abstract

The power grid is a complex and vital system that necessitates careful reliability management. Managing the grid is a difficult problem with multiple time scales of decision making and stochastic behavior due to renewable energy generations, variable demand and unplanned outages. Solving this problem in the face of uncertainty requires a new methodology with tractable algorithms. In this work, we introduce a new model for hierarchical decision making in complex systems. We apply reinforcement learning (RL) methods to learn a proxy, i.e., a level of abstraction, for real-time power grid reliability. We devise an algorithm that alternates between slow time-scale policy improvement, and fast time-scale value function approximation. We compare our results to prevailing heuristics, and show the strength of our method.