The Two Boundaries: Why Behavioral AI Governance Fails Structurally

TL;DR

This paper analyzes the structural limitations of behavioral AI governance, demonstrating that effective governance requires architectural separation of effects from computation, and introduces coterminous governance as a key criterion.

Contribution

It provides a formal framework showing the inherent undecidability of behavioral effect governance and proposes coterminous governance as a necessary architectural design.

Findings

Rice's theorem implies the undecidability of effect compliance in general.

Coterminous governance requires separating computation from effects.

Proofs of the framework are mechanized in Coq with 454 theorems.

Abstract

Every system that performs effects has two boundaries: what it can do (expressiveness) and what governance covers (governance). In nearly all deployed AI systems, these boundaries are defined independently, creating three regions: governed capabilities (the only useful region), ungoverned capabilities (risk), and governance policies that address non-existent capabilities (theater). Two of the three regions are failure modes. We focus on the governance of effects: actions that AI systems perform in the world (API calls, database writes, tool invocations). This is distinct from the governance of model outputs (content quality, bias, fairness), which operates at a different level and requires different mechanisms. We present a formal framework for analyzing this structural gap. Rice's theorem (1953) proves the gap is undecidable in the general case for any Turing-complete architecture that attempts to govern effects behaviorally: no algorithm can decide non-trivial semantic properties of arbitrary programs, including the property "this program's effects comply with the governance policy." We define coterminous governance: a system property where the expressivenessboundary equals the governance boundary. We show that coterminous governance requires an architectural decision (separatingcomputation from effect) rather than a governance layer added after the fact. We show that structural governance under this separation subsumes separate governance infrastructure: governance checks become part of the execution pipeline rather than a second system running alongside it. We propose coterminous governance as the testable criterion for any AI governance system: either the two boundaries are provably identical, or risk and theater are structurally inevitable. Proofs are mechanized in Coq (454 theorems, 36 modules, 0 admitted).