Reinforcement Learning-Based Air Traffic Deconfliction

TL;DR

This paper presents a reinforcement learning approach for automating horizontal aircraft separation to ensure safety in congested airspace, validated through high-fidelity simulation and real-world demonstration.

Contribution

It introduces a novel RL-based system for aircraft deconfliction that translates learned policies into executable trajectories in real-time.

Findings

System generates quick, safe avoidance trajectories

Validated in high-fidelity simulation

Successfully demonstrated in full-scale airplane test

Abstract

Remain Well Clear, keeping the aircraft away from hazards by the appropriate separation distance, is an essential technology for the safe operation of uncrewed aerial vehicles in congested airspace. This work focuses on automating the horizontal separation of two aircraft and presents the obstacle avoidance problem as a 2D surrogate optimization task. By our design, the surrogate task is made more conservative to guarantee the execution of the solution in the primary domain. Using Reinforcement Learning (RL), we optimize the avoidance policy and model the dynamics, interactions, and decision-making. By recursively sampling the resulting policy and the surrogate transitions, the system translates the avoidance policy into a complete avoidance trajectory. Then, the solver publishes the trajectory as a set of waypoints for the airplane to follow using the Robot Operating System (ROS) interface. The proposed system generates a quick and achievable avoidance trajectory that satisfies the safety requirements. Evaluation of our system is completed in a high-fidelity simulation and full-scale airplane demonstration. Moreover, the paper concludes an enormous integration effort that has enabled a real-life demonstration of the RL-based system.