HUGO -- Highlighting Unseen Grid Options: Combining Deep Reinforcement Learning with a Heuristic Target Topology Approach

TL;DR

This paper introduces a holistic DRL approach for power grid management by incorporating robust target topologies, significantly improving performance and survival times in automated grid operation tasks.

Contribution

It proposes a novel search algorithm for robust target topologies and upgrades a DRL agent to incorporate these topologies, enhancing grid operation performance.

Findings

10% increase in L2RPN score with topology actions

25% improvement in median survival time

Most target topologies are close to the base topology, indicating robustness.

Abstract

With the growth of Renewable Energy (RE) generation, the operation of power grids has become increasingly complex. One solution could be automated grid operation, where Deep Reinforcement Learning (DRL) has repeatedly shown significant potential in Learning to Run a Power Network (L2RPN) challenges. However, only individual actions at the substation level have been subjected to topology optimization by most existing DRL algorithms. In contrast, we propose a more holistic approach by proposing specific Target Topologies (TTs) as actions. These topologies are selected based on their robustness. As part of this paper, we present a search algorithm to find the TTs and upgrade our previously developed DRL agent CurriculumAgent (CAgent) to a novel topology agent. We compare the upgrade to the previous CAgent and can increase their L2RPN score significantly by 10%. Further, we achieve a 25% better median survival time with our TTs included. Later analysis shows that almost all TTs are close to the base topology, explaining their robustness