Preventive Energy Management for Distribution Systems Under Uncertain Events: A Deep Reinforcement Learning Approach

TL;DR

This paper introduces a deep reinforcement learning-based preventive energy management system that accounts for uncertainties and failures in distribution systems, enhancing resilience and operational efficiency.

Contribution

It proposes a CVaR-based framework combined with PPO reinforcement learning to optimize energy management under uncertainty in distribution systems.

Findings

PPO-based RL effectively optimizes system performance under uncertainties.

The framework improves system resilience compared to traditional methods.

Validated on MVDC ship and IEEE 30-bus systems.

Abstract

As power systems become more complex with the continuous integration of intelligent distributed energy resources (DERs), new risks and uncertainties arise. Consequently, to enhance system resiliency, it is essential to account for various uncertain events when implementing the optimization problem for the energy management system (EMS). This paper presents a preventive EMS considering the probability of failure (PoF) of each system component across different scenarios. A conditional-value-at-risk (CVaR)-based framework is proposed to integrate the uncertainties of the distribution network. Loads are classified into critical, semi-critical, and non-critical categories to prioritize essential loads during generation resource shortages. A proximal policy optimization (PPO)-based reinforcement learning (RL) agent is used to solve the formulated problem and generate the control decisions. The proposed framework is evaluated on a notional MVDC ship system and a modified IEEE 30-bus system, where the results demonstrate that the PPO agent can successfully optimize the objective function while maintaining the network and operational constraints. For validation, the RL-based method is benchmarked against a traditional optimization approach, further highlighting its effectiveness and robustness. This comparison shows that RL agents can offer more resiliency against future uncertain events compared to the traditional solution methods due to their adaptability and learning capacity.