Artificial General Intelligence, Existential Risk, and Human Risk Perception

TL;DR

This paper discusses the rising perception of existential risks from artificial general intelligence (AGI), highlighting that both experts and non-experts see it as a significant threat, with perceptions increasing rapidly in recent years.

Contribution

It provides an analysis of risk perception related to AGI using publicly available forecaster and opinion data, revealing consensus and uncertainties among different groups.

Findings

Perceived risk of AGI as a catastrophic threat exceeds other risks.

Perception of AGI risk has increased steeply over the past year.

Consensus exists on AGI being a pressing existential risk, but its basis is unclear.

Abstract

Artificial general intelligence (AGI) does not yet exist, but given the pace of technological development in artificial intelligence, it is projected to reach human-level intelligence within roughly the next two decades. After that, many experts expect it to far surpass human intelligence and to do so rapidly. The prospect of superintelligent AGI poses an existential risk to humans because there is no reliable method for ensuring that AGI goals stay aligned with human goals. Drawing on publicly available forecaster and opinion data, the author examines how experts and non-experts perceive risk from AGI. The findings indicate that the perceived risk of a world catastrophe or extinction from AGI is greater than for other existential risks. The increase in perceived risk over the last year is also steeper for AGI than for other existential threats (e.g., nuclear war or human-caused climate change). That AGI is a pressing existential risk is something on which experts and non-experts agree, but the basis for such agreement currently remains obscure.