Applications of Reinforcement Learning in Deregulated Power Market: A Comprehensive Review

TL;DR

This paper reviews how reinforcement learning techniques are increasingly used to optimize bidding and dispatching strategies in deregulated power markets, addressing challenges like uncertainty and computational efficiency.

Contribution

It provides a comprehensive overview of RL applications in deregulated power markets, including methodologies, challenges, and future potential, based on over 150 studies.

Findings

RL effectively improves bidding strategies.

RL addresses uncertainty and computational challenges.

Potential RL techniques for deployment are identified.

Abstract

The increasing penetration of renewable generations, along with the deregulation and marketization of power industry, promotes the transformation of power market operation paradigms. The optimal bidding strategy and dispatching methodology under these new paradigms are prioritized concerns for both market participants and power system operators, with obstacles of uncertain characteristics, computational efficiency, as well as requirements of hyperopic decision-making. To tackle these problems, the Reinforcement Learning (RL), as an emerging machine learning technique with advantages compared with conventional optimization tools, is playing an increasingly significant role in both academia and industry. This paper presents a comprehensive review of RL applications in deregulated power market operation including bidding and dispatching strategy optimization, based on more than 150 carefully selected literatures. For each application, apart from a paradigmatic summary of generalized methodology, in-depth discussions of applicability and obstacles while deploying RL techniques are also provided. Finally, some RL techniques that have great potentiality to be deployed in bidding and dispatching problems are recommended and discussed.