Verifying International Agreements on AI: Six Layers of Verification for Rules on Large-Scale AI Development and Deployment

TL;DR

This paper proposes a comprehensive six-layer verification framework for international oversight of large-scale AI development, emphasizing technical, personnel, and security measures to ensure compliance and manage risks.

Contribution

It introduces a novel multi-layer verification framework with detailed implementation options and highlights key R&D challenges for effective international AI compliance oversight.

Findings

Six largely independent verification approaches identified

Verification methods require guardrails to prevent abuse

Many verification technologies are still under development

Abstract

The risks of frontier AI may require international cooperation, which in turn may require verification: checking that all parties follow agreed-on rules. For instance, states might need to verify that powerful AI models are widely deployed only after their risks to international security have been evaluated and deemed manageable. However, research on AI verification could benefit from greater clarity and detail. To address this, this report provides an in-depth overview of AI verification, intended for both policy professionals and technical researchers. We present novel conceptual frameworks, detailed implementation options, and key R&D challenges. These draw on existing literature, expert interviews, and original analysis, all within the scope of confidentially overseeing AI development and deployment that uses thousands of high-end AI chips. We find that states could eventually verify compliance by using six largely independent verification approaches with substantial redundancy: (1) built-in security features in AI chips; (2-3) separate monitoring devices attached to AI chips; and (4-6) personnel-based mechanisms, such as whistleblower programs. While promising, these approaches require guardrails to protect against abuse and power concentration, and many of these technologies have yet to be built or stress-tested. To enable states to confidently verify compliance with rules on large-scale AI development and deployment, the R&D challenges we list need significant progress.