Designing an Objective-Driven Test Method for the Comparative Performance Evaluation of Commercial DTI Solutions for Counter UAS systems

TL;DR

This paper introduces an objective, scenario-based test methodology for benchmarking commercial DTI solutions in counter-UAS systems, enabling consistent performance evaluation in operational environments.

Contribution

It develops and validates a standardized, scenario-driven testing framework for DTI systems, addressing the lack of existing benchmarks and enabling component and system-level performance comparison.

Findings

Method effectively evaluates detection, tracking, and identification components.

Validation conducted through three operational trials.

Enables performance comparison in realistic environments.

Abstract

Unmanned Aerial Systems (UASs) or drones become more and more commercially available and cheap. There has been much emphasis on developing and deploying Counter-UAS systems (UASs) with Detection Tracking and Identification (DTI) solutions. However, the capabilities of these systems are hard to benchmark. Performance claims of these systems are currently not supported by evidence. In addition, no standard test methodologies are available for these DTI systems and different test methodologies make comparison of these systems hard or impossible. We report on the definition, development and verification of an objective-driven test method and corresponding comparative performance evaluation for commercial DTI solutions for C-UASs. The developed methodology is based on end-user scenarios that are operationally relevant. The test methodology is based on a generic DTI system lay-out and is detailed towards detection, tracking and identification, taking into account contextual information and end-user input. The comparative performance evaluation is developed to enable the use of the methodology in a relevant environment, thereby taking into account any potential environmental aspect that might influence DTI system performance. Validation of the work in a relevant environment has been done in three operational trials. The operational trial results show that the method allows for performance evaluation at component level (i.e., detection, tracking or identification component) and at system level (combinations of these components and integrated DTI system of system solutions).