Multi-Objective Reinforcement Learning for Power Grid Topology Control

TL;DR

This paper applies multi-objective reinforcement learning to power grid topology control, enabling better trade-offs among conflicting objectives and improving grid reliability compared to single-objective methods.

Contribution

It introduces a novel MORL approach using DOL and MOPPO for power grid topology control, effectively balancing multiple operational objectives.

Findings

MORL policies are 30% more successful in preventing grid failure.

MORL improves Pareto front approximation over random search.

Multi-objective RL outperforms single-objective RL when training budget is limited.

Abstract

Transmission grid congestion increases as the electrification of various sectors requires transmitting more power. Topology control, through substation reconfiguration, can reduce congestion but its potential remains under-exploited in operations. A challenge is modeling the topology control problem to align well with the objectives and constraints of operators. Addressing this challenge, this paper investigates the application of multi-objective reinforcement learning (MORL) to integrate multiple conflicting objectives for power grid topology control. We develop a MORL approach using deep optimistic linear support (DOL) and multi-objective proximal policy optimization (MOPPO) to generate a set of Pareto-optimal policies that balance objectives such as minimizing line loading, topological deviation, and switching frequency. Initial case studies show that the MORL approach can provide valuable insights into objective trade-offs and improve Pareto front approximation compared to a random search baseline. The generated multi-objective RL policies are 30% more successful in preventing grid failure under contingencies and 20% more effective when training budget is reduced - compared to the common single objective RL policy.