Adam-based Augmented Random Search for Control Policies for Distributed Energy Resource Cyber Attack Mitigation

TL;DR

This paper introduces an Adam-based augmented random search method to quickly learn control policies for non-compromised distributed energy resources, enhancing cybersecurity response in power systems.

Contribution

It presents a novel ARS-based approach for DER control policy learning that outperforms traditional deep reinforcement learning in speed and efficiency.

Findings

ARS learns policies an order of magnitude faster than DRL.

The method effectively mitigates voltage oscillations during cyber-attacks.

Control policies can be linear or complex, depending on system needs.

Abstract

Volt-VAR and Volt-Watt control functions are mechanisms that are included in distributed energy resource (DER) power electronic inverters to mitigate excessively high or low voltages in distribution systems. In the event that a subset of DER have had their Volt-VAR and Volt-Watt settings compromised as part of a cyber-attack, we propose a mechanism to control the remaining set of non-compromised DER to ameliorate large oscillations in system voltages and large voltage imbalances in real time. To do so, we construct control policies for individual non-compromised DER, directly searching the policy space using an Adam-based augmented random search (ARS). In this paper we show that, compared to previous efforts aimed at training policies for DER cybersecurity using deep reinforcement learning (DRL), the proposed approach is able to learn optimal (and sometimes linear) policies an order of magnitude faster than conventional DRL techniques (e.g., Proximal Policy Optimization).