Are we Doomed to an AI Race? Why Self-Interest Could Drive Countries Towards a Moratorium on Superintelligence

TL;DR

This paper employs game theory to demonstrate that self-interested countries might find it rational to agree on a moratorium on AI superintelligence development, considering risks and strategic benefits.

Contribution

It formalizes the strategic interactions between nations regarding ASI and shows how increasing perceived risks can incentivize a global moratorium.

Findings

A moratorium can be in a country's self-interest under certain risk perceptions.

Perceived costs of losing control influence countries' incentives to halt ASI development.

Empirical evidence indicates rising global concern about ASI risks.

Abstract

This paper uses game theory to argue that, contrary to the prevailing view, a moratorium on Artificial Superintelligence (ASI) can be in a state's self-interest. By formalizing trategic interactions between geopolitical superpowers, we model the trade-off between the benefits of technological supremacy and the catastrophic risks of uncontrolled ASI. The analysis reveals that as the perceived cost of loss of control increases sufficiently relative to other parameters, it becomes in each state's self-interest to impose a moratorium. We further provide empirical evidence suggesting that the global perception of ASI risk is rising, making a stable, rational moratorium increasingly plausible in the current geopolitical landscape.