Emergent Behaviors in Multi-Agent Target Acquisition

TL;DR

This paper investigates agent behaviors in multi-agent systems using reinforcement learning in pursuit-evasion scenarios, introducing a novel feature set for behavior classification to improve coordination and target prediction.

Contribution

It presents a new approach for classifying agent behaviors in multi-agent systems by using heatmaps and feature sets, aiding in behavior prediction and coordination.

Findings

Heatmaps effectively categorize agent behaviors.

The feature set reveals data regularities for behavior classification.

Behavior classification can improve target catching and agent coordination.

Abstract

Only limited studies and superficial evaluations are available on agents' behaviors and roles within a Multi-Agent System (MAS). We simulate a MAS using Reinforcement Learning (RL) in a pursuit-evasion (a.k.a predator-prey pursuit) game, which shares task goals with target acquisition, and we create different adversarial scenarios by replacing RL-trained pursuers' policies with two distinct (non-RL) analytical strategies. Using heatmaps of agents' positions (state-space variable) over time, we are able to categorize an RL-trained evader's behaviors. The novelty of our approach entails the creation of an influential feature set that reveals underlying data regularities, which allow us to classify an agent's behavior. This classification may aid in catching the (enemy) targets by enabling us to identify and predict their behaviors, and when extended to pursuers, this approach towards identifying teammates' behavior may allow agents to coordinate more effectively.