A Secure Learning Control Strategy via Dynamic Camouflaging for Unknown Dynamical Systems under Attacks

TL;DR

This paper introduces a dynamic camouflaging reinforcement learning approach that enhances security in unknown cyber-physical systems by preventing attackers from accurately learning system dynamics during control.

Contribution

It proposes a novel ARRL algorithm that simultaneously learns optimal control and injects misinformation to thwart attackers' learning efforts, with theoretical guarantees and extensive experiments.

Findings

The ARRL algorithm effectively learns control policies under attack scenarios.

The method successfully misleads attackers during the learning process.

Numerical experiments demonstrate robustness on multi-agent and power grid systems.

Abstract

This paper presents a secure reinforcement learning (RL) based control method for unknown linear time-invariant cyber-physical systems (CPSs) that are subjected to compositional attacks such as eavesdropping and covert attack. We consider the attack scenario where the attacker learns about the dynamic model during the exploration phase of the learning conducted by the designer to learn a linear quadratic regulator (LQR), and thereafter, use such information to conduct a covert attack on the dynamic system, which we refer to as doubly learning-based control and attack (DLCA) framework. We propose a dynamic camouflaging based attack-resilient reinforcement learning (ARRL) algorithm which can learn the desired optimal controller for the dynamic system, and at the same time, can inject sufficient misinformation in the estimation of system dynamics by the attacker. The algorithm is accompanied by theoretical guarantees and extensive numerical experiments on a consensus multi-agent system and on a benchmark power grid model.