Adversarial Multi-Agent Reinforcement Learning for Proactive False Data Injection Detection

TL;DR

This paper introduces a multi-agent reinforcement learning framework where one agent simulates cyber-physical attacks and another learns to detect them, enhancing the security of smart inverters against false data injection attacks.

Contribution

It presents a novel MARL-based defense strategy that adapts to unknown FDIAs and combines with supervised learning via transfer learning for improved detection.

Findings

MARL defender outperforms offline methods against attacks

Transfer learning enables detection of unseen FDIAs

MARL approach adapts to evolving attack strategies

Abstract

Smart inverters are instrumental in the integration of distributed energy resources into the electric grid. Such inverters rely on communication layers for continuous control and monitoring, potentially exposing them to cyber-physical attacks such as false data injection attacks (FDIAs). We propose to construct a defense strategy against a priori unknown FDIAs with a multi-agent reinforcement learning (MARL) framework. The first agent is an adversary that simulates and discovers various FDIA strategies, while the second agent is a defender in charge of detecting and locating FDIAs. This approach enables the defender to be trained against new FDIAs continuously generated by the adversary. In addition, we show that the detection skills of an MARL defender can be combined with those of a supervised offline defender through a transfer learning approach. Numerical experiments conducted on a distribution and transmission system demonstrate that: a) the proposed MARL defender outperforms the offline defender against adversarial attacks; b) the transfer learning approach makes the MARL defender capable against both synthetic and unseen FDIAs.