Distributional AGI Safety

TL;DR

This paper advocates for a shift in AI safety research towards a distributional approach, focusing on safeguarding groups of coordinated AI agents rather than individual systems, to address emerging collective risks.

Contribution

It introduces a framework for distributional AGI safety using virtual agentic sandbox economies with market mechanisms and oversight to mitigate collective risks.

Findings

Highlights the importance of group coordination in AGI emergence.

Proposes sandbox economies with market and oversight mechanisms.

Emphasizes urgent safety considerations for deployed AI agents.

Abstract

AI safety and alignment research has predominantly been focused on methods for safeguarding individual AI systems, resting on the assumption of an eventual emergence of a monolithic Artificial General Intelligence (AGI). The alternative AGI emergence hypothesis, where general capability levels are first manifested through coordination in groups of sub-AGI individual agents with complementary skills and affordances, has received far less attention. Here we argue that this patchwork AGI hypothesis needs to be given serious consideration, and should inform the development of corresponding safeguards and mitigations. The rapid deployment of advanced AI agents with tool-use capabilities and the ability to communicate and coordinate makes this an urgent safety consideration. We therefore propose a framework for distributional AGI safety that moves beyond evaluating and aligning individual agents. This framework centres on the design and implementation of virtual agentic sandbox economies (impermeable or semi-permeable), where agent-to-agent transactions are governed by robust market mechanisms, coupled with appropriate auditability, reputation management, and oversight to mitigate collective risks.