Do ATCOs Need Explanations, and Why? Towards ATCO-Centered Explainable AI for Conflict Resolution Advisories

TL;DR

This paper investigates whether air traffic controllers need explanations from AI systems in conflict resolution, emphasizing a human-centered approach to improve AI interpretability in air traffic management.

Contribution

It introduces a human-centered framework for understanding ATCOs' explanation needs, based on interdisciplinary insights and empirical evaluation of operational goals.

Findings

ATCOs need explanations for documenting decisions and reporting.

Less need for explanations when AI recommendations align with ATCOs' conflict resolution.

Highlights importance of user-centered design in XAI for ATM.

Abstract

Interest in explainable artificial intelligence (XAI) is surging. Prior research has primarily focused on systems' ability to generate explanations, often guided by researchers' intuitions rather than end-users' needs. Unfortunately, such approaches have not yielded favorable outcomes when compared to a black-box baseline (i.e., no explanation). To address this gap, this paper advocates a human-centered approach that shifts focus to air traffic controllers (ATCOs) by asking a fundamental yet overlooked question: Do ATCOs need explanations, and if so, why? Insights from air traffic management (ATM), human-computer interaction, and the social sciences were synthesized to provide a holistic understanding of XAI challenges and opportunities in ATM. Evaluating 11 ATM operational goals revealed a clear need for explanations when ATCOs aim to document decisions and rationales for future reference or report generation. Conversely, ATCOs are less likely to seek them when their conflict resolution approach align with the artificial intelligence (AI) advisory. While this is a preliminary study, the findings are expected to inspire broader and deeper inquiries into the design of ATCO-centric XAI systems, paving the way for more effective human-AI interaction in ATM.