Human Agency, Causality, and the Human Computer Interface in High-Stakes Artificial Intelligence

TL;DR

This paper emphasizes the importance of preserving human causal control in high-stakes AI interfaces, critiquing current XAI approaches and proposing a new Causal-Agency Framework to enhance human agency and understanding.

Contribution

It introduces a nested Causal-Agency Framework that integrates causal models and uncertainty to improve human-centered AI interfaces and preserve human agency.

Findings

Current XAI focuses on correlation, neglecting causality and uncertainty.

AI interfaces can distort human perception of system causality, leading to errors.

The proposed CAF aims to restore human causal control in high-stakes AI systems.

Abstract

Current discourse on Artificial Intelligence (AI) ethics, dominated by "trustworthy" and "responsible" AI, overlooks a more fundamental human-computer interaction (HCI) crisis: the erosion of human agency. This paper argues that the primary challenge of high-stakes AI systems is not trust, but the preservation of human causal control. We posit that "bad AI" will function as "bad UI," a metaphor for catastrophic interface failures that misrepresent system state and lead to human error. Applying Marshall McLuhan's media theory, AI can be framed as a technology of "augmentation" that simultaneously "amputates" the user's direct perception of causality. This places the interface as the critical locus where a "double uncertainty"--that of the human user and that of the probabilistic model--must be mediated. We critique current Explainable AI (XAI) for its correlational focus and failure to represent uncertainty. We conclude by proposing a rigorous, nested Causal-Agency Framework (CAF) that integrates causal models, uncertainty quantification, and human-centered evaluation to restore agency at the interface.