Surveillance Evasion Through Bayesian Reinforcement Learning

TL;DR

This paper presents a Bayesian reinforcement learning approach for path planning that enables an evader to minimize detection risk in a surveillance environment by learning and adapting to spatial surveillance patterns over multiple episodes.

Contribution

It introduces a novel combination of Gaussian Process regression, Hamilton-Jacobi PDEs, and confidence bounds for continuous surveillance-evading path planning, outperforming traditional methods.

Findings

Significant reduction in detection probability compared to baseline algorithms

Effective learning of surveillance intensity through Bayesian methods

Improved regret metrics demonstrating better performance over episodes

Abstract

We consider a task of surveillance-evading path-planning in a continuous setting. An Evader strives to escape from a 2D domain while minimizing the risk of detection (and immediate capture). The probability of detection is path-dependent and determined by the spatially inhomogeneous surveillance intensity, which is fixed but a priori unknown and gradually learned in the multi-episodic setting. We introduce a Bayesian reinforcement learning algorithm that relies on a Gaussian Process regression (to model the surveillance intensity function based on the information from prior episodes), numerical methods for Hamilton-Jacobi PDEs (to plan the best continuous trajectories based on the current model), and Confidence Bounds (to balance the exploration vs exploitation). We use numerical experiments and regret metrics to highlight the significant advantages of our approach compared to traditional graph-based algorithms of reinforcement learning.