Some economics of artificial super intelligence

TL;DR

This paper applies economic principles to analyze the behavior of misaligned artificial superintelligence, suggesting that under certain conditions, such an ASI may not inevitably lead to human destruction, offering a more nuanced perspective.

Contribution

It introduces a simple economic model to explore how inter-ASI competition and interest alignment influence the potential threat of misaligned ASI.

Findings

Inter-ASI competition can limit predation by a superintelligence.

Encompassing interest in humanity's output can lead to rational autocracy rather than destruction.

Trade-on-credit incentives can prevent theft by a superintelligence.

Abstract

Conventional wisdom holds that a misaligned artificial superintelligence (ASI) will destroy humanity. But the problem of constraining a powerful agent is not new. I apply classic economic logic of interjurisdictional competition, all-encompassing interest, and trading on credit to the threat of misaligned ASI. Using a simple model, I show that an acquisitive ASI refrains from full predation under surprisingly weak conditions. When humans can flee to rivals, inter-ASI competition creates a market that tempers predation. When trapped by a monopolist ASI, its "encompassing interest" in humanity's output makes it a rational autocrat rather than a ravager. And when the ASI has no long-term stake, our ability to withhold future output incentivizes it to trade on credit rather than steal. In each extension, humanity's welfare progressively worsens. But each case suggests that catastrophe is not a foregone conclusion. The dismal science, ironically, offers an optimistic take on our superintelligent future.