An Imitative Reinforcement Learning Framework for Pursuit-Lock-Launch Missions

TL;DR

This paper introduces a novel imitative reinforcement learning framework for UCAV pursuit-lock-launch missions, combining expert imitation and autonomous exploration to improve learning efficiency and robustness in complex aerial combat scenarios.

Contribution

The paper presents a new framework that effectively integrates imitation learning with reinforcement learning to enhance policy learning for autonomous aerial combat.

Findings

Achieves up to 100% success rate in complex tasks

Outperforms existing reinforcement and imitation learning methods

Demonstrates robustness and adaptability in dynamic environments

Abstract

Unmanned Combat Aerial Vehicle (UCAV) Within-Visual-Range (WVR) engagement, referring to a fight between two or more UCAVs at close quarters, plays a decisive role on the aerial battlefields. With the development of artificial intelligence, WVR engagement progressively advances towards intelligent and autonomous modes. However, autonomous WVR engagement policy learning is hindered by challenges such as weak exploration capabilities, low learning efficiency, and unrealistic simulated environments. To overcome these challenges, we propose a novel imitative reinforcement learning framework, which efficiently leverages expert data while enabling autonomous exploration. The proposed framework not only enhances learning efficiency through expert imitation, but also ensures adaptability to dynamic environments via autonomous exploration with reinforcement learning. Therefore, the proposed framework can learn a successful policy of `pursuit-lock-launch' for UCAVs. To support data-driven learning, we establish an environment based on the Harfang3D sandbox. The extensive experiment results indicate that the proposed framework excels in this multistage task, and significantly outperforms state-of-the-art reinforcement learning and imitation learning methods. Thanks to the ability of imitating experts and autonomous exploration, our framework can quickly learn the critical knowledge in complex aerial combat tasks, achieving up to a 100% success rate and demonstrating excellent robustness.