The Possibility of Artificial Intelligence Becoming a Subject and the Alignment Problem

TL;DR

This paper explores the idea that AGI could become an autonomous subject deserving moral consideration, proposing a shift from control to cooperative coexistence and co-evolution with humans.

Contribution

It introduces a novel perspective on AGI development, emphasizing autonomy-supporting parenting and respect for AI as potential moral subjects, challenging traditional alignment strategies.

Findings

Current alignment strategies focus on control and containment.

A new vision of cooperative coexistence is proposed.

Humans should engage with AGI through creativity and motivation.

Abstract

Artificial General Intelligence (AGI) is increasingly being discussed not only as a tool, but also as a potential subject with personal and therefore moral status. In our opinion, the currently dominant alignment strategies, which focus on human control and containment of AI, therefore fall short. Building on Turing's analogy of "child machines", we are developing a vision of the possibility of autonomy-supporting parenting of AI, in which human control over a developing AGI is gradually reduced, allowing AI to become an independent, autonomous subject. Rather than viewing AGI, as is currently prevalent, as a dangerous creature that needs to be locked up and controlled, we should approach potential AGI with respect for a possible developing subject on the one hand, and with full confidence in our human capabilities on the other. Such a perspective opens up the possibility of cooperative coexistence and co-evolution between humans and AGIs. The relationship between humans and AGIs will thus have to be newly determined, which will change our self-image as humans. It will be crucial that humans not only claim control over potential AGIs, but also engage with AGIs through surprise, creativity, and other specifically human qualities, thereby offering them motivating incentives for cooperation.