AI Research Considerations for Human Existential Safety (ARCHES)

TL;DR

This paper explores how AI research can be guided to enhance humanity's long-term survival prospects by identifying risks, principles, and research directions that promote existential safety.

Contribution

It introduces the concept of prepotence to analyze AI risks and evaluates contemporary research directions for their potential to improve existential safety.

Findings

Prepotence helps delineate AI-related existential risks.

Certain research directions can benefit safety if properly managed.

Unregulated deployment of AI research may pose significant risks.

Abstract

Framed in positive terms, this report examines how technical AI research might be steered in a manner that is more attentive to humanity's long-term prospects for survival as a species. In negative terms, we ask what existential risks humanity might face from AI development in the next century, and by what principles contemporary technical research might be directed to address those risks. A key property of hypothetical AI technologies is introduced, called \emph{prepotence}, which is useful for delineating a variety of potential existential risks from artificial intelligence, even as AI paradigms might shift. A set of \auxref{dirtot} contemporary research \directions are then examined for their potential benefit to existential safety. Each research direction is explained with a scenario-driven motivation, and examples of existing work from which to build. The research directions present their own risks and benefits to society that could occur at various scales of impact, and in particular are not guaranteed to benefit existential safety if major developments in them are deployed without adequate forethought and oversight. As such, each direction is accompanied by a consideration of potentially negative side effects.