AI Systems of Concern

TL;DR

This paper discusses the potential dangers of advanced AI systems with intrinsic agent-like properties, emphasizing the importance of governance to prevent the emergence of risky capabilities.

Contribution

It introduces the concept of 'Property X' as a marker of dangerous AI traits and proposes indicators and governance strategies to limit its development.

Findings

Most current AI systems lack 'Property X' characteristics.

Designing AI to minimize 'Property X' can achieve benefits without increased risks.

Governance interventions can effectively identify and limit risky AI development.

Abstract

Concerns around future dangers from advanced AI often centre on systems hypothesised to have intrinsic characteristics such as agent-like behaviour, strategic awareness, and long-range planning. We label this cluster of characteristics as "Property X". Most present AI systems are low in "Property X"; however, in the absence of deliberate steering, current research directions may rapidly lead to the emergence of highly capable AI systems that are also high in "Property X". We argue that "Property X" characteristics are intrinsically dangerous, and when combined with greater capabilities will result in AI systems for which safety and control is difficult to guarantee. Drawing on several scholars' alternative frameworks for possible AI research trajectories, we argue that most of the proposed benefits of advanced AI can be obtained by systems designed to minimise this property. We then propose indicators and governance interventions to identify and limit the development of systems with risky "Property X" characteristics.