Examining the Differential Risk from High-level Artificial Intelligence and the Question of Control

TL;DR

This paper models AI risk using a hierarchical complex systems framework, highlighting uncertainties in AI impact, control challenges, and potential for catastrophic failures as AI systems grow more autonomous.

Contribution

It introduces a hierarchical complex systems model for AI risk assessment and provides empirical survey data on expert opinions about AI impact and safety.

Findings

Increased uncertainty about powerful AI agent scenarios

High confidence in multiagent environment dynamics

Growing concern over AI alignment failures and influence-seeking behaviors

Abstract

Artificial Intelligence (AI) is one of the most transformative technologies of the 21st century. The extent and scope of future AI capabilities remain a key uncertainty, with widespread disagreement on timelines and potential impacts. As nations and technology companies race toward greater complexity and autonomy in AI systems, there are concerns over the extent of integration and oversight of opaque AI decision processes. This is especially true in the subfield of machine learning (ML), where systems learn to optimize objectives without human assistance. Objectives can be imperfectly specified or executed in an unexpected or potentially harmful way. This becomes more concerning as systems increase in power and autonomy, where an abrupt capability jump could result in unexpected shifts in power dynamics or even catastrophic failures. This study presents a hierarchical complex systems framework to model AI risk and provide a template for alternative futures analysis. Survey data were collected from domain experts in the public and private sectors to classify AI impact and likelihood. The results show increased uncertainty over the powerful AI agent scenario, confidence in multiagent environments, and increased concern over AI alignment failures and influence-seeking behavior.