Realising Synthetic Active Inference Agents, Part I: Epistemic Objectives and Graphical Specification Language

TL;DR

This paper develops a graphical framework for active inference agents based on free energy minimization, introducing a new notation to specify constraints and deriving algorithms for policy inference, demonstrated on a T-maze task.

Contribution

It introduces Constrained Forney-style Factor Graphs (CFFG) for graphical specification of variational objectives and derives new algorithms for active inference, including direct policy inference.

Findings

Developed CFFG notation for graphical variational inference

Derived algorithms enabling direct policy inference for AIF agents

Demonstrated information seeking behavior on T-maze task

Abstract

The Free Energy Principle (FEP) is a theoretical framework for describing how (intelligent) systems self-organise into coherent, stable structures by minimising a free energy functional. Active Inference (AIF) is a corollary of the FEP that specifically details how systems that are able to plan for the future (agents) function by minimising particular free energy functionals that incorporate information seeking components. This paper is the first in a series of two where we derive a synthetic version of AIF on free form factor graphs. The present paper focuses on deriving a local version of the free energy functionals used for AIF. This enables us to construct a version of AIF which applies to arbitrary graphical models and interfaces with prior work on message passing algorithms. The resulting messages are derived in our companion paper. We also identify a gap in the graphical notation used for factor graphs. While factor graphs are great at expressing a generative model, they have so far been unable to specify the full optimisation problem including constraints. To solve this problem we develop Constrained Forney-style Factor Graph (CFFG) notation which permits a fully graphical description of variational inference objectives. We then proceed to show how CFFG's can be used to reconstruct prior algorithms for AIF as well as derive new ones. The latter is demonstrated by deriving an algorithm that permits direct policy inference for AIF agents, circumventing a long standing scaling issue that has so far hindered the application of AIF in industrial settings. We demonstrate our algorithm on the classic T-maze task and show that it reproduces the information seeking behaviour that is a hallmark feature of AIF.