Fully Autonomous AI Agents Should Not be Developed

TL;DR

This paper argues against the development of fully autonomous AI agents, highlighting increased safety and ethical risks as autonomy levels rise, and emphasizes the importance of cautious AI development.

Contribution

It provides a detailed analysis of AI agent levels, ethical considerations, and trade-offs, emphasizing the risks associated with higher autonomy.

Findings

Risks to people increase with AI autonomy.

Safety risks are particularly concerning.

Higher autonomy leads to greater ethical trade-offs.

Abstract

This paper argues that fully autonomous AI agents should not be developed. In support of this position, we build from prior scientific literature and current product marketing to delineate different AI agent levels and detail the ethical values at play in each, documenting trade-offs in potential benefits and risks. Our analysis reveals that risks to people increase with the autonomy of a system: The more control a user cedes to an AI agent, the more risks to people arise. Particularly concerning are safety risks, which affect human life and impact further values.