AI Consciousness and Existential Risk

TL;DR

This paper clarifies the distinction between AI consciousness and intelligence, emphasizing that only intelligence directly relates to existential risk, while consciousness may influence risk indirectly in specific scenarios.

Contribution

It disentangles the concepts of AI consciousness and intelligence, highlighting their empirical and theoretical differences and implications for AI safety and policy.

Findings

Intelligence predicts existential risk, consciousness does not.

Consciousness may influence risk indirectly through specific scenarios.

Clarifying these distinctions aids AI safety research and policymaking.

Abstract

In AI, the existential risk denotes the hypothetical threat posed by an artificial system that would possess both the capability and the objective, either directly or indirectly, to eradicate humanity. This issue is gaining prominence in scientific debate due to recent technical advancements and increased media coverage. In parallel, AI progress has sparked speculation and studies about the potential emergence of artificial consciousness. The two questions, AI consciousness and existential risk, are sometimes conflated, as if the former entailed the latter. Here, I explain that this view stems from a common confusion between consciousness and intelligence. Yet these two properties are empirically and theoretically distinct. Arguably, while intelligence is a direct predictor of an AI system's existential threat, consciousness is not. There are, however, certain incidental scenarios in which consciousness could influence existential risk, in either direction. Consciousness could be viewed as a means towards AI alignment, thereby lowering existential risk; or, it could be a precondition for reaching certain capabilities or levels of intelligence, and thus positively related to existential risk. Recognizing these distinctions can help AI safety researchers and public policymakers focus on the most pressing issues.