Explainable Interface for Human-Autonomy Teaming: A Survey

TL;DR

This survey explores the development and evaluation of Explainable Interfaces in human-autonomy teaming, emphasizing transparency, trust, and human-centric design in safety-critical applications using XAI techniques.

Contribution

It introduces a structured framework for Explainable Interfaces in HAT, clarifies key concepts, and provides a comprehensive evaluation approach based on extensive interdisciplinary surveys.

Findings

Proposed a novel framework for EI in HAT systems.

Identified key challenges and design principles for EI.

Outlined future research directions in XAI-enhanced HAT.

Abstract

Nowadays, large-scale foundation models are being increasingly integrated into numerous safety-critical applications, including human-autonomy teaming (HAT) within transportation, medical, and defence domains. Consequently, the inherent 'black-box' nature of these sophisticated deep neural networks heightens the significance of fostering mutual understanding and trust between humans and autonomous systems. To tackle the transparency challenges in HAT, this paper conducts a thoughtful study on the underexplored domain of Explainable Interface (EI) in HAT systems from a human-centric perspective, thereby enriching the existing body of research in Explainable Artificial Intelligence (XAI). We explore the design, development, and evaluation of EI within XAI-enhanced HAT systems. To do so, we first clarify the distinctions between these concepts: EI, explanations and model explainability, aiming to provide researchers and practitioners with a structured understanding. Second, we contribute to a novel framework for EI, addressing the unique challenges in HAT. Last, our summarized evaluation framework for ongoing EI offers a holistic perspective, encompassing model performance, human-centered factors, and group task objectives. Based on extensive surveys across XAI, HAT, psychology, and Human-Computer Interaction (HCI), this review offers multiple novel insights into incorporating XAI into HAT systems and outlines future directions.