Deep Reinforcement Learning based Model-free On-line Dynamic Multi-Microgrid Formation to Enhance Resilience

TL;DR

This paper introduces a deep reinforcement learning framework for real-time, dynamic multi-microgrid formation to improve power system resilience, utilizing topology transformation and CNN-based Q-networks for efficient decision-making.

Contribution

It presents a novel deep RL approach with topology transformation and action-decoupling for online multi-microgrid formation, enhancing system resilience.

Findings

Effective in 7-bus system and IEEE 123-bus system

Provides real-time, adaptive response to changing conditions

Significantly improves system resilience

Abstract

Multi-microgrid formation (MMGF) is a promising solution to enhance power system resilience. This paper proposes a new deep reinforcement learning (RL) based model-free on-line dynamic multi-MG formation (MMGF) scheme. The dynamic MMGF problem is formulated as a Markov decision process, and a complete deep RL framework is specially designed for the topology-transformable micro-grids. In order to reduce the large action space caused by flexible switch operations, a topology transformation method is proposed and an action-decoupling Q-value is applied. Then, a CNN based multi-buffer double deep Q-network (CM-DDQN) is developed to further improve the learning ability of original DQN method. The proposed deep RL method provides real-time computing to support on-line dynamic MMGF scheme, and the scheme handles a long-term resilience enhancement problem using adaptive on-line MMGF to defend changeable conditions. The effectiveness of the proposed method is validated using a 7-bus system and the IEEE 123-bus system. The results show strong learning ability, timely response for varying system conditions and convincing resilience enhancement.