Sparse Wide-Area Control of Power Systems using Data-driven Reinforcement Learning

TL;DR

This paper introduces a data-driven reinforcement learning approach for sparse wide-area control in power systems, enabling robust oscillation damping with reduced communication costs despite model uncertainties.

Contribution

It develops a reinforcement learning-based method that learns sparse controllers online, improving stability and efficiency in uncertain power system models.

Findings

The method achieves reliable LQR performance in uncertain conditions.

Sparse controllers reduce communication costs in distributed implementation.

The approach outperforms nominal model-based controllers under severe uncertainties.

Abstract

In this paper we present an online wide-area oscillation damping control (WAC) design for uncertain models of power systems using ideas from reinforcement learning. We assume that the exact small-signal model of the power system at the onset of a contingency is not known to the operator and use the nominal model and online measurements of the generator states and control inputs to rapidly converge to a state-feedback controller that minimizes a given quadratic energy cost. However, unlike conventional linear quadratic regulators (LQR), we intend our controller to be sparse, so its implementation reduces the communication costs. We, therefore, employ the gradient support pursuit (GraSP) optimization algorithm to impose sparsity constraints on the control gain matrix during learning. The sparse controller is thereafter implemented using distributed communication. Using the IEEE 39-bus power system model with 1149 unknown parameters, it is demonstrated that the proposed learning method provides reliable LQR performance while the controller matched to the nominal model becomes unstable for severely uncertain systems.