Research on Autonomous Maneuvering Decision of UCAV based on Approximate Dynamic Programming

TL;DR

This paper develops an approximate dynamic programming method enabling UCAVs to make real-time, autonomous maneuvering decisions in complex air combat scenarios by learning strategies through reinforcement learning without explicit rules.

Contribution

It introduces a novel approximate dynamic programming framework with feature selection and function approximation for autonomous UCAV maneuvering in 3D air combat environments.

Findings

Effective in real-time decision making

Handles high-dimensional combat scenarios

Learns strategies without explicit rules

Abstract

Unmanned aircraft systems can perform some more dangerous and difficult missions than manned aircraft systems. In some highly complicated and changeable tasks, such as air combat, the maneuvering decision mechanism is required to sense the combat situation accurately and make the optimal strategy in real-time. This paper presents a formulation of a 3-D one-on-one air combat maneuvering problem and an approximate dynamic programming approach for computing an optimal policy on autonomous maneuvering decision making. The aircraft learns combat strategies in a Reinforcement Leaning method, while sensing the environment, taking available maneuvering actions and getting feedback reward signals. To solve the problem of dimensional explosion in the air combat, the proposed method is implemented through feature selection, trajectory sampling, function approximation and Bellman backup operation in the air combat simulation environment. This approximate dynamic programming approach provides a fast response to a rapidly changing tactical situation, learns in long-term planning, without any explicitly coded air combat rule base.