The Controllability Trap: A Governance Framework for Military AI Agents

TL;DR

This paper introduces the Agentic Military AI Governance Framework (AMAGF), a structured approach to maintaining human control over autonomous military AI systems through real-time metrics and multi-layered governance.

Contribution

It proposes a novel governance architecture with measurable control metrics and concrete mechanisms to address control failures in agentic military AI systems.

Findings

Control Quality Score (CQS) enables real-time control assessment

Framework formalizes responsibilities across five institutional actors

Operational scenario demonstrates practical implementation

Abstract

Agentic AI systems - capable of goal interpretation, world modeling, planning, tool use, long-horizon operation, and autonomous coordination - introduce distinct control failures not addressed by existing safety frameworks. We identify six agentic governance failures tied to these capabilities and show how they erode meaningful human control in military settings. We propose the Agentic Military AI Governance Framework (AMAGF), a measurable architecture structured around three pillars: Preventive Governance (reducing failure likelihood), Detective Governance (real-time detection of control degradation), and Corrective Governance (restoring or safely degrading operations). Its core mechanism, the Control Quality Score (CQS), is a composite real-time metric quantifying human control and enabling graduated responses as control weakens. For each failure type, we define concrete mechanisms, assign responsibilities across five institutional actors, and formalize evaluation metrics. A worked operational scenario illustrates implementation, and we situate the framework within established agent safety literature. We argue that governance must move from a binary conception of control to a continuous model in which control quality is actively measured and managed throughout the operational lifecycle.