Game-Theory-Assisted Reinforcement Learning for Border Defense: Early Termination based on Analytical Solutions

TL;DR

This paper presents a hybrid game-theoretic and reinforcement learning approach for border defense, using analytical solutions for early episode termination to improve training efficiency and performance.

Contribution

It introduces a novel method combining game theory and RL with early termination based on the Apollonius Circle, enhancing training efficiency in complex border defense scenarios.

Findings

10-20% higher rewards in experiments

Faster convergence of RL training

More efficient search trajectories

Abstract

Game theory provides the gold standard for analyzing adversarial engagements, offering strong optimality guarantees. However, these guarantees often become brittle when assumptions such as perfect information are violated. Reinforcement learning (RL), by contrast, is adaptive but can be sample-inefficient in large, complex domains. This paper introduces a hybrid approach that leverages game-theoretic insights to improve RL training efficiency. We study a border defense game with limited perceptual range, where defender performance depends on both search and pursuit strategies, making classical differential game solutions inapplicable. Our method employs the Apollonius Circle (AC) to compute equilibrium in the post-detection phase, enabling early termination of RL episodes without learning pursuit dynamics. This allows RL to concentrate on learning search strategies while guaranteeing optimal continuation after detection. Across single- and multi-defender settings, this early termination method yields 10-20% higher rewards, faster convergence, and more efficient search trajectories. Extensive experiments validate these findings and demonstrate the overall effectiveness of our approach.