Adversarial Online Learning with Variable Plays in the Pursuit-Evasion Game: Theoretical Foundations and Application in Connected and Automated Vehicle Cybersecurity

TL;DR

This paper develops a theoretical framework for adversarial pursuit-evasion games with variable resource allocation, applying it to cybersecurity in connected vehicles, and proposes algorithms with proven regret bounds.

Contribution

It extends multi-armed bandit models to variable plays in pursuit-evasion games, providing new theoretical insights and algorithms for cybersecurity applications.

Findings

Existence of Nash equilibrium under certain conditions

Proposed exponential-weighted algorithm with sublinear regret

Demonstrated effectiveness of variable-arm play through experiments

Abstract

We extend the adversarial/non-stochastic multi-play multi-armed bandit (MPMAB) to the case where the number of arms to play is variable. The work is motivated by the fact that the resources allocated to scan different critical locations in an interconnected transportation system change dynamically over time and depending on the environment. By modeling the malicious hacker and the intrusion monitoring system as the attacker and the defender, respectively, we formulate the problem for the two players as a sequential pursuit-evasion game. We derive the condition under which a Nash equilibrium of the strategic game exists. For the defender side, we provide an exponential-weighted based algorithm with sublinear pseudo-regret. We further extend our model to heterogeneous rewards for both players, and obtain lower and upper bounds on the average reward for the attacker. We provide numerical experiments to demonstrate the effectiveness of a variable-arm play.