Safety Must Precede the Deployment of Open-Ended AI

TL;DR

This paper emphasizes that safety considerations must be prioritized before deploying open-ended AI systems, due to unique risks like unpredictability and emergent misalignment that differ from traditional models.

Contribution

It highlights the distinct safety challenges of open-ended AI and advocates for proactive research and coordinated efforts to ensure safe deployment.

Findings

Open-ended AI introduces safety risks like unpredictability and misalignment.

Existing safety frameworks may be insufficient for open-ended systems.

Preemptive safety research is crucial before large-scale deployment.

Abstract

AI advancements have been significantly driven by a combination of foundation models and curiosity-driven learning aimed at increasing capability and adaptability. Within this landscape, open-endedness, where AI agents autonomously and indefinitely generate novel behaviors, representations, or solutions, has gained increasing interest. This has become relevant in the context of self-evolving agents and long-horizon discovery. This position paper argues that the defining properties of open-ended AI systems introduce a distinct and underexplored class of safety challenges, including loss of predictability, emergent misalignment, and difficulties in maintaining effective control as systems evolve beyond their initial design assumptions, that must be addressed preemptively. These challenges differ qualitatively from those associated with task-bounded or static models and are unlikely to be addressed by existing safety frameworks alone, which is why these risks must be examined proactively, before large-scale deployment. The paper outlines key challenges, discusses research opportunities, and calls for coordinated action to support the safe and responsible development of open-ended AI.