Model-Free RL Agents Demonstrate System 1-Like Intentionality

TL;DR

This paper demonstrates that model-free reinforcement learning agents, despite lacking explicit planning, exhibit behaviors akin to human System 1 thinking, challenging assumptions about intentionality requiring planning.

Contribution

It introduces a novel framework linking System 1 and System 2 cognition to model-free and model-based RL, expanding understanding of AI intentionality.

Findings

Model-free RL agents show reactive, intentional behaviors.

The framework links cognitive psychology to RL agent behavior.

Implications for AI responsibility and safety are discussed.

Abstract

This paper argues that model-free reinforcement learning (RL) agents, while lacking explicit planning mechanisms, exhibit behaviours that can be analogised to System 1 ("thinking fast") processes in human cognition. Unlike model-based RL agents, which operate akin to System 2 ("thinking slow") reasoning by leveraging internal representations for planning, model-free agents react to environmental stimuli without anticipatory modelling. We propose a novel framework linking the dichotomy of System 1 and System 2 to the distinction between model-free and model-based RL. This framing challenges the prevailing assumption that intentionality and purposeful behaviour require planning, suggesting instead that intentionality can manifest in the structured, reactive behaviours of model-free agents. By drawing on interdisciplinary insights from cognitive psychology, legal theory, and experimental jurisprudence, we explore the implications of this perspective for attributing responsibility and ensuring AI safety. These insights advocate for a broader, contextually informed interpretation of intentionality in RL systems, with implications for their ethical deployment and regulation.