Evader-Agnostic Team-Based Pursuit Strategies in Partially-Observable Environments

TL;DR

This paper introduces a neuro-symbolic, two-phase pursuit strategy for UAV teams in urban environments, combining offline deep reinforcement learning with online opponent classification to improve pursuit success against unknown evaders.

Contribution

It presents a novel two-phase approach integrating deep reinforcement learning and opponent classification for pursuit-evasion in partially observable environments.

Findings

Improved average pursuit success against random evaders.

Effective two-phase strategy combining offline training and online adaptation.

Demonstrated applicability in urban UAV pursuit scenarios.

Abstract

We consider a scenario where a team of two unmanned aerial vehicles (UAVs) pursue an evader UAV within an urban environment. Each agent has a limited view of their environment where buildings can occlude their field-of-view. Additionally, the pursuer team is agnostic about the evader in terms of its initial and final location, and the behavior of the evader. Consequently, the team needs to gather information by searching the environment and then track it to eventually intercept. To solve this multi-player, partially-observable, pursuit-evasion game, we develop a two-phase neuro-symbolic algorithm centered around the principle of bounded rationality. First, we devise an offline approach using deep reinforcement learning to progressively train adversarial policies for the pursuer team against fictitious evaders. This creates $k$-levels of rationality for each agent in preparation for the online phase. Then, we employ an online classification algorithm to determine a "best guess" of our current opponent from the set of iteratively-trained strategic agents and apply the best player response. Using this schema, we improved average performance when facing a random evader in our environment.