Predictive Minds: LLMs As Atypical Active Inference Agents

TL;DR

This paper reinterprets large language models through the lens of active inference, highlighting their current limitations and potential for future self-aware, adaptive behavior driven by feedback loops.

Contribution

It introduces a novel perspective by framing LLMs as atypical active inference agents, contrasting them with traditional systems and discussing future enhancements.

Findings

LLMs currently lack a tight feedback loop between action and perception.

LLMs fit within the active inference paradigm despite current limitations.

Closing the feedback loop could lead to self-aware, adaptive models.

Abstract

Large language models (LLMs) like GPT are often conceptualized as passive predictors, simulators, or even stochastic parrots. We instead conceptualize LLMs by drawing on the theory of active inference originating in cognitive science and neuroscience. We examine similarities and differences between traditional active inference systems and LLMs, leading to the conclusion that, currently, LLMs lack a tight feedback loop between acting in the world and perceiving the impacts of their actions, but otherwise fit in the active inference paradigm. We list reasons why this loop may soon be closed, and possible consequences of this including enhanced model self-awareness and the drive to minimize prediction error by changing the world.