Omniscient Attacker in Stochastic Security Games with Interdependent Nodes

TL;DR

This paper investigates the vulnerability of reinforcement learning-based defenses in stochastic security games, revealing that an omniscient attacker can exploit learning dynamics to outperform naive defenses, emphasizing the need for more robust strategies.

Contribution

It extends the analysis of attacker vulnerabilities from normal-form to stochastic security games using a linear influence network model and neuro-dynamic programming.

Findings

Omniscient attacker outperforms naive defender in stochastic security games.

Learning dynamics introduce critical vulnerabilities in RL-based defense strategies.

Proposed neuro-dynamic programming approach effectively models attacker strategies.

Abstract

The adoption of reinforcement learning for critical infrastructure defense introduces a vulnerability where sophisticated attackers can strategically exploit the defense algorithm's learning dynamics. While prior work addresses this vulnerability in the context of repeated normal-form games, its extension to the stochastic games remains an open research gap. We close this gap by examining stochastic security games between an RL defender and an omniscient attacker, utilizing a tractable linear influence network model. To overcome the structural limitations of prior methods, we propose and apply neuro-dynamic programming. Our experimental results demonstrate that the omniscient attacker can significantly outperform a naive defender, highlighting the critical vulnerability introduced by the learning dynamics and the effectiveness of the proposed strategy.