Fusion of Model-free Reinforcement Learning with Microgrid Control: Review and Vision

TL;DR

This paper reviews the integration of model-free reinforcement learning into microgrid control, highlighting current methods, challenges, and future research directions for enhancing distributed energy resource management.

Contribution

It provides a comprehensive overview of how MFRL can be fused with microgrid control frameworks, including methodologies, applications, and challenges, offering a structured research map.

Findings

MFRL algorithms can be integrated via model identification, signal generation, or controller substitution.

Challenges include stability, data efficiency, and real-time implementation of MFRL in microgrids.

The paper outlines future research directions for effective MFRL deployment in microgrid control.

Abstract

Challenges and opportunities coexist in microgrids as a result of emerging large-scale distributed energy resources (DERs) and advanced control techniques. In this paper, a comprehensive review of microgrid control is presented with its fusion of model-free reinforcement learning (MFRL). A high-level research map of microgrid control is developed from six distinct perspectives, followed by bottom-level modularized control blocks illustrating the configurations of grid-following (GFL) and grid-forming (GFM) inverters. Then, mainstream MFRL algorithms are introduced with an explanation of how MFRL can be integrated into the existing control framework. Next, the application guideline of MFRL is summarized with a discussion of three fusing approaches, i.e., model identification and parameter tuning, supplementary signal generation, and controller substitution, with the existing control framework. Finally, the fundamental challenges associated with adopting MFRL in microgrid control and corresponding insights for addressing these concerns are fully discussed.