Analyzing the Closed-Loop Performance of Detect-And-Avoid Systems

TL;DR

This paper evaluates the closed-loop safety and efficiency of Detect-And-Avoid systems in unmanned aircraft, revealing significant inefficiencies in dense traffic and limitations of neural network approximations for complex scenarios.

Contribution

It provides the first comprehensive analysis of closed-loop DAA performance, highlighting safety-efficiency trade-offs and computational challenges in simulation.

Findings

High inefficiency in dense airspaces.

Neural networks struggle with multiple intruders.

Safety correlates with open-loop deviation indicators.

Abstract

Detect-And-Avoid (DAA) algorithms for unmanned air vehicles have industry standards called Minimum Operational Performance Standards (MOPS), establishing criteria to check whether they can ensure safe separation for all plausible operational conditions. However, these MOPS ensure performance for the avoidance maneuvers, which are open-loop, but not for the maneuvers that bring the air vehicles back to their intended courses, closing the control loop of the missions. In this paper, we analyze the closed-loop performance of existing DAA algorithms, by experimenting large numbers of traffic configurations with 4 aircraft in a delimited airspace. We measure and analyze their rates of loss of separation and timeout events, the latter happening when a chain of maneuvers exceeds the maximum supply of energy in a vehicle. We also analyze the efficiency of the closed-loop logic, expressed as the excess fuel rate, and study the relationship between safety and efficiency in these scenarios. Our results show that the inefficiency caused by DAA algorithms is very significant in dense airspaces and that it offers ample space for improvement. Performing the simulations of the closed-loop mission management logic can be highly time consuming. Despite there being Neural Network approximations of DAA logic that alleviate the computational load, none of those that we found available can properly handle cases with more than one intruder in the ownship surveillance range. Therefore, in an attempt to overcome the high computational cost of analyzing the closed-loop performance of DAA algorithms, we study the correlation between inefficiency and safety of closed-loop scenarios, and some indicators from the corresponding open-loop scenarios, such as angle of deviation and number of deviations per flight time.