Verification methods for international AI agreements

TL;DR

This paper reviews 10 verification techniques for international AI agreements, focusing on detecting unauthorized AI training and data centers, and categorizes them based on access requirements and hardware dependencies.

Contribution

It introduces a comprehensive categorization of verification methods, analyzes their effectiveness, and discusses potential evasion techniques and future research directions.

Findings

National technical means can detect violations with minimal access.

Access-dependent methods require cooperation from suspected nations.

Hardware-dependent methods rely on hardware rules to ensure compliance.

Abstract

What techniques can be used to verify compliance with international agreements about advanced AI development? In this paper, we examine 10 verification methods that could detect two types of potential violations: unauthorized AI training (e.g., training runs above a certain FLOP threshold) and unauthorized data centers. We divide the verification methods into three categories: (a) national technical means (methods requiring minimal or no access from suspected non-compliant nations), (b) access-dependent methods (methods that require approval from the nation suspected of unauthorized activities), and (c) hardware-dependent methods (methods that require rules around advanced hardware). For each verification method, we provide a description, historical precedents, and possible evasion techniques. We conclude by offering recommendations for future work related to the verification and enforcement of international AI governance agreements.