Reinforcement Learning based Air Combat Maneuver Generation

TL;DR

This paper explores the use of reinforcement learning, specifically TD3 and HER, to generate optimal air combat maneuvers for UAVs with Dubins vehicle dynamics in simulated environments, demonstrating improved decision-making.

Contribution

The study applies advanced RL algorithms to UAV maneuver generation, integrating HER for experience replay, and evaluates performance in simulated air combat scenarios.

Findings

RL-based UAV maneuvering outperforms traditional methods in simulations

TD3 combined with HER effectively learns optimal paths in 2D space

Reinforcement learning shows promise against human pilots in simulated air combat

Abstract

The advent of artificial intelligence technology paved the way of many researches to be made within air combat sector. Academicians and many other researchers did a research on a prominent research direction called autonomous maneuver decision of UAV. Elaborative researches produced some outcomes, but decisions that include Reinforcement Learning(RL) came out to be more efficient. There have been many researches and experiments done to make an agent reach its target in an optimal way, most prominent are Genetic Algorithm(GA) , A star, RRT and other various optimization techniques have been used. But Reinforcement Learning is the well known one for its success. In DARPHA Alpha Dogfight Trials, reinforcement learning prevailed against a real veteran F16 human pilot who was trained by Boeing. This successor model was developed by Heron Systems. After this accomplishment, reinforcement learning bring tremendous attention on itself. In this research we aimed our UAV which has a dubin vehicle dynamic property to move to the target in two dimensional space in an optimal path using Twin Delayed Deep Deterministic Policy Gradients (TD3) and used in experience replay Hindsight Experience Replay(HER).We did tests on two different environments and used simulations.