Analyzing Errors in Controlled Turret System Given Target Location Input from Artificial Intelligence Methods in Automatic Target Recognition

TL;DR

This study evaluates how AI-based object detection errors affect the accuracy of a controlled turret system in targeting stationary objects, revealing key metrics that predict system performance.

Contribution

It provides a comprehensive analysis of the impact of AI detection errors on turret accuracy and identifies metrics that predict successful targeting.

Findings

AI errors significantly impact turret accuracy

Confidence score and IoU are predictive of hit probability

Metrics like average precision and recall correlate with targeting success

Abstract

In this paper, we assess the movement error of a targeting system given target location data from artificial intelligence (AI) methods in automatic target recognition (ATR) systems. Few studies evaluate the impacts on the accuracy in moving a targeting system to an aimpoint provided in this manner. To address this knowledge gap, we assess the performance of a controlled gun turret system given target location from an object detector developed from AI methods. In our assessment, we define a measure of object detector error and examine the correlations with several standard metrics in object detection. We then statistically analyze the object detector error data and turret movement error data acquired from controlled targeting simulations, as well as their aggregate error, to examine the impact on turret movement accuracy. Finally, we study the correlations between additional metrics and the probability of a hit. The results indicate that AI technologies are a significant source of error to targeting systems. Moreover, the results suggest that metrics such as the confidence score, intersection-over-union, average precision and average recall are predictors of accuracy against stationary targets with our system parameters.