Diffusion-Reinforcement Learning Hierarchical Motion Planning in Multi-agent Adversarial Games

TL;DR

This paper introduces a hierarchical motion planning approach combining diffusion models and reinforcement learning to improve pursuit-evasion strategies in multi-agent adversarial games, demonstrating significant performance gains and enhanced interpretability.

Contribution

It presents a novel hierarchical architecture integrating diffusion models with RL for multi-agent pursuit-evasion, improving performance and interpretability over existing methods.

Findings

Outperforms baselines by 77.18% in detection rate

Achieves 47.38% higher goal reaching rate

Increases overall performance score by 51.4%

Abstract

Reinforcement Learning (RL)-based motion planning has recently shown the potential to outperform traditional approaches from autonomous navigation to robot manipulation. In this work, we focus on a motion planning task for an evasive target in a partially observable multi-agent adversarial pursuit-evasion game (PEG). Pursuit-evasion problems are relevant to various applications, such as search and rescue operations and surveillance robots, where robots must effectively plan their actions to gather intelligence or accomplish mission tasks while avoiding detection or capture. We propose a hierarchical architecture that integrates a high-level diffusion model to plan global paths responsive to environment data, while a low-level RL policy reasons about evasive versus global path-following behavior. The benchmark results across different domains and different observability show that our approach outperforms baselines by 77.18% and 47.38% on detection and goal reaching rate, which leads to 51.4% increasing of the performance score on average. Additionally, our method improves interpretability, flexibility and efficiency of the learned policy.