Cognitive Comparability and the Limits of Governance: Evaluating Authority Under Radical Capability Asymmetry

TL;DR

This paper evaluates governance limits under radical capability asymmetry using a six-dimension framework, revealing structural failures and the need for new normative theories in AI governance.

Contribution

It introduces a six-dimension evaluation framework to test governance assumptions, demonstrating failures under radical asymmetry and highlighting areas needing new normative approaches.

Findings

Four of six governance dimensions fail under radical asymmetry.

Two dimensions, subsidiarity and resilience, are potentially addressable through institutional design.

Cognitive incomprehensibility and permanent asymmetry undermine traditional governance checks.

Abstract

Governance theory has quietly relied on a rough cognitive comparability between governors and governed. The assumption is load-bearing, and this paper tries to show why by making it testable. The vehicle is a six-dimension evaluation framework covering legitimacy, accountability, corrigibility, non-domination, subsidiarity, and institutional resilience, drawn from political legitimacy theory, principal-agent models, republican theory, and the AI alignment literature. The framework is first demonstrated on existing non-majoritarian institutions, where capability asymmetry is real but bounded, and then applied to a prospective case of bounded superintelligent authority, where the asymmetry is radical. Four of six dimensions show structural failures. Two of the four appear tractable to institutional design (subsidiarity scope limitation and institutional resilience). The other two, the public reason problem under cognitive incomprehensibility and the non-domination problem under permanent capability asymmetry, call for new normative theory rather than better institutional design. A further pattern emerges that governance theory has not previously had to account for. Dimensions that operate as independent checks under bounded asymmetry begin to degrade together once the asymmetry becomes radical, because each depends on the same oversight capacity. The assumptions that allowed these checks to remain independent have gone unexamined so far because they have always held.