Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges

TL;DR

This paper reviews recent advances in reinforcement learning applications for power system management, focusing on frequency regulation, voltage control, and energy management, highlighting challenges and future research directions.

Contribution

It provides a comprehensive overview of RL techniques in power systems and discusses key issues and future directions for their application.

Findings

RL techniques are increasingly used in power system control.

Key challenges include safety, robustness, scalability, and data issues.

Future research directions are identified for RL in power systems.

Abstract

With large-scale integration of renewable generation and distributed energy resources, modern power systems are confronted with new operational challenges, such as growing complexity, increasing uncertainty, and aggravating volatility. Meanwhile, more and more data are becoming available owing to the widespread deployment of smart meters, smart sensors, and upgraded communication networks. As a result, data-driven control techniques, especially reinforcement learning (RL), have attracted surging attention in recent years. This paper provides a comprehensive review of various RL techniques and how they can be applied to decision-making and control in power systems. In particular, we select three key applications, i.e., frequency regulation, voltage control, and energy management, as examples to illustrate RL-based models and solutions. We then present the critical issues in the application of RL, i.e., safety, robustness, scalability, and data. Several potential future directions are discussed as well.