An International Agreement to Prevent the Premature Creation of Artificial Superintelligence

TL;DR

This paper proposes an international agreement led by the US and China to prevent premature development of artificial superintelligence, focusing on verification, restrictions, and safety to mitigate catastrophic risks.

Contribution

It introduces a framework for global AI governance that combines verification, legal restrictions, and cooperation to delay dangerous AI development.

Findings

The proposed agreement could technically prevent ASI development if implemented today.

Verification methods include FLOP thresholds and chip tracking.

Political will remains a major obstacle to implementation.

Abstract

Many experts argue that premature development of artificial superintelligence (ASI) poses catastrophic risks, including the risk of human extinction from misaligned ASI, geopolitical instability, and misuse by malicious actors. This report proposes an international agreement to prevent the premature development of ASI until AI development can proceed without these risks. The agreement halts dangerous AI capabilities advancement while preserving access to current, safe AI applications. The proposed framework centers on a coalition led by the United States and China that would restrict the scale of AI training and dangerous AI research. Due to the lack of trust between parties, verification is a key part of the agreement. Limits on the scale of AI training are operationalized by FLOP thresholds and verified through the tracking of AI chips and verification of chip use. Dangerous AI research--that which advances toward artificial superintelligence or endangers the agreement's verifiability--is stopped via legal prohibitions and multifaceted verification. We believe the proposal would be technically sufficient to forestall the development of ASI if implemented today, but advancements in AI capabilities or development methods could hurt its efficacy. Additionally, there does not yet exist the political will to put such an agreement in place. Despite these challenges, we hope this agreement can provide direction for AI governance research and policy.