Generator Parameter Estimation by Q-Learning Based on PMU Measurements

TL;DR

This paper introduces a Q-learning based method for estimating synchronous generator parameters using PMU data, employing event playback and an optimal exploration policy to improve estimation accuracy.

Contribution

It presents a novel Q-learning approach that uses event playback and history-dependent policies for generator parameter estimation from PMU measurements.

Findings

Effective parameter estimation demonstrated on a synchronous generator model

Improved computational efficiency through sequential event utilization

Q-learning approach outperforms traditional estimation methods

Abstract

In this paper, a novel Q-learning based approach is proposed for estimating the parameters of synchronous generators using PMU measurements. Event playback is used to generate model outputs under different parameters for training the agent in Q-learning. We assume that the exact values of some parameters in the model are not known by the agent in Q-learning. Then, an optimal history-dependent policy for the exploration-exploitation trade-off is planned. With given prior knowledge, the parameter vector can be viewed as states with a specific reward, which is a function of the fitting error compared with the measurements. The agent takes an action (either increasing or decreasing the parameter) and the estimated parameter will move to a new state. Based on the reward function, the optimal action policy will move the parameter set to a state with the highest reward. If multiple events are available, they will be used sequentially so that the updated $\mathbfcal{Q}$-value can be utilized to improve the computational efficiency. The effectiveness of the proposed approach is validated by estimating the parameters of the dynamic model of a synchronous generator.