Reinforcement Learning for Power-Flow Network Analysis

TL;DR

This paper introduces a reinforcement learning approach to estimate and find multiple solutions of power flow equations in electrical networks, surpassing traditional algebraic methods in complexity and scale.

Contribution

It develops a probabilistic reward function and a state-space model for RL agents to approximate and discover multiple solutions of non-linear power flow equations.

Findings

RL agents find more solutions than the average baseline

The method scales to larger networks than traditional algebraic algorithms

The approach demonstrates potential for complex non-linear algebra problems

Abstract

The power flow equations are non-linear multivariate equations that describe the relationship between power injections and bus voltages of electric power networks. Given a network topology, we are interested in finding network parameters with many equilibrium points. This corresponds to finding instances of the power flow equations with many real solutions. Current state-of-the art algorithms in computational algebra are not capable of answering this question for networks involving more than a small number of variables. To remedy this, we design a probabilistic reward function that gives a good approximation to this root count, and a state-space that mimics the space of power flow equations. We derive the average root count for a Gaussian model, and use this as a baseline for our RL agents. The agents discover instances of the power flow equations with many more solutions than the average baseline. This demonstrates the potential of RL for power-flow network design and analysis as well as the potential for RL to contribute meaningfully to problems that involve complex non-linear algebra or geometry. \footnote{Author order alphabetic, all authors contributed equally.