ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning

TL;DR

This paper introduces ARM-IRL, a data-driven, adaptive resilience metric learning method for cyber-physical systems using inverse reinforcement learning, enabling more accurate resilience assessment amid changing system states.

Contribution

It develops a novel inverse reinforcement learning approach to learn resilience metrics dynamically, improving upon static or fuzzy logic-based methods.

Findings

Successfully applied to network rerouting and reconfiguration scenarios

Outperforms static metrics in adapting to system changes

Demonstrates effectiveness on IEEE 123-bus system

Abstract

Resilience of safety-critical systems is gaining importance, particularly with the increasing number of cyber and physical threats. Cyber-physical threats are becoming increasingly prevalent, as digital systems are ubiquitous in critical infrastructure. The challenge with determining the resilience of cyber-physical systems is identifying a set of resilience metrics that can adapt to the changing states of the system. A static resilience metric can lead to an inaccurate estimation of system state, and can result in unintended consequences against cyber threats. In this work, we propose a data-driven method for adaptive resilience metric learning. The primary goal is to learn a single resilience metric by formulating an inverse reinforcement learning problem that learns a reward or objective from a set of control actions from an expert. It learns the structure or parameters of the reward function based on information provided by expert demonstrations. Most prior work has considered static weights or theories from fuzzy logic to formulate a single resilience metric. Instead, this work learns the resilience metric, represented as reward function, using adversarial inverse reinforcement learning, to determine the optimal policy through training the generator discriminator in parallel. We evaluate our proposed technique in scenarios such as optimal communication network rerouting, power distribution network reconfiguration, and a combined cyber-physical restoration of critical load using the IEEE 123-bus system.