Incompleteness of AI Safety Verification via Kolmogorov Complexity

TL;DR

This paper demonstrates that AI safety verification faces fundamental, information-theoretic limitations, showing that no finite verifier can certify all high-complexity policy-compliant instances due to Kolmogorov complexity.

Contribution

It formalizes the verification problem using Kolmogorov complexity and proves an incompleteness result highlighting intrinsic limits of AI safety verification.

Findings

No finite verifier can certify all high-complexity policy instances.

Verification limitations are rooted in information-theoretic, not just computational, constraints.

Motivates proof-carrying approaches for instance-level guarantees.

Abstract

Ensuring that artificial intelligence (AI) systems satisfy formal safety and policy constraints is a central challenge in safety-critical domains. While limitations of verification are often attributed to combinatorial complexity and model expressiveness, we show that they arise from intrinsic information-theoretic limits. We formalize policy compliance as a verification problem over encoded system behaviors and analyze it using Kolmogorov complexity. We prove an incompleteness result: for any fixed sound computably enumerable verifier, there exists a threshold beyond which true policy-compliant instances cannot be certified once their complexity exceeds that threshold. Consequently, no finite formal verifier can certify all policy-compliant instances of arbitrarily high complexity. This reveals a fundamental limitation of AI safety verification independent of computational resources, and motivates proof-carrying approaches that provide instance-level correctness guarantees.