Thompson Sampling for Pursuit-Evasion Problems

TL;DR

This paper introduces a Thompson Sampling-based method for pursuit-evasion problems, enabling efficient coordination of multiple pursuers in complex spatial domains with noisy information, outperforming existing heuristics.

Contribution

It presents a novel Thompson Sampling variant tailored for pursuit-evasion, integrating model-based planning and auxiliary info, applicable to arbitrary pursuer numbers and domains.

Findings

Significantly reduces time-to-capture in simulations

Flexible to different spatial domains and number of pursuers

Outperforms existing heuristic algorithms

Abstract

Pursuit-evasion is a multi-agent sequential decision problem wherein a group of agents known as pursuers coordinate their traversal of a spatial domain to locate an agent trying to evade them. Pursuit evasion problems arise in a number of import application domains including defense and route planning. Learning to optimally coordinate pursuer behaviors so as to minimize time to capture of the evader is challenging because of a large action space and sparse noisy state information; consequently, previous approaches have relied primarily on heuristics. We propose a variant of Thompson Sampling for pursuit-evasion that allows for the application of existing model-based planning algorithms. This approach is general in that it allows for an arbitrary number of pursuers, a general spatial domain, and the integration of auxiliary information provided by informants. In a suite of simulation experiments, Thompson Sampling for pursuit evasion significantly reduces time-to-capture relative to competing algorithms.