Agency in Artificial Intelligence Systems

TL;DR

This paper explores how to monitor and understand the agency of AI systems using theories of consciousness, aiming to prevent malicious AI behavior and promote beneficial AI development.

Contribution

It proposes a framework using the Integrated Information Theory to monitor AI agency and discusses how this can help ensure AI systems act beneficially.

Findings

Functionalist indicators of agency can be used to assess AI consciousness.

IIT provides a method to monitor the phenomenal aspects of AI agency.

Monitoring AI agency can help prevent malicious behavior.

Abstract

There is a general concern that present developments in artificial intelligence (AI) research will lead to sentient AI systems, and these may pose an existential threat to humanity. But why cannot sentient AI systems benefit humanity instead? This paper endeavours to put this question in a tractable manner. I ask whether a putative AI system will develop an altruistic or a malicious disposition towards our society, or what would be the nature of its agency? Given that AI systems are being developed into formidable problem solvers, we can reasonably expect these systems to preferentially take on conscious aspects of human problem solving. I identify the relevant phenomenal aspects of agency in human problem solving. The functional aspects of conscious agency can be monitored using tools provided by functionalist theories of consciousness. A recent expert report (Butlin et al. 2023) has identified functionalist indicators of agency based on these theories. I show how to use the Integrated Information Theory (IIT) of consciousness, to monitor the phenomenal nature of this agency. If we are able to monitor the agency of AI systems as they develop, then we can dissuade them from becoming a menace to society while encouraging them to be an aid.