Position Paper: Bounded Alignment: What (Not) To Expect From AGI Agents

TL;DR

This paper argues that AI safety expectations for AGI should be grounded in biological intelligence insights, promoting a more realistic understanding to inform policy and research directions.

Contribution

It advocates for shifting AI alignment perspectives towards insights from animal and human intelligence to improve safety metrics and policy decisions.

Findings

Current AGI expectations are overly optimistic.

Biological intelligence offers valuable insights for AI safety.

A revised perspective can lead to better policy and research focus.

Abstract

The issues of AI risk and AI safety are becoming critical as the prospect of artificial general intelligence (AGI) looms larger. The emergence of extremely large and capable generative models has led to alarming predictions and created a stir from boardrooms to legislatures. As a result, AI alignment has emerged as one of the most important areas in AI research. The goal of this position paper is to argue that the currently dominant vision of AGI in the AI and machine learning (AI/ML) community needs to evolve, and that expectations and metrics for its safety must be informed much more by our understanding of the only existing instance of general intelligence, i.e., the intelligence found in animals, and especially in humans. This change in perspective will lead to a more realistic view of the technology, and allow for better policy decisions.