Branching Time Active Inference: the theory and its generality

TL;DR

This paper introduces a unified framework for active inference planning using tree search algorithms that propagate expected free energy forward and backward, clarifying their relation to active inference and sophisticated inference.

Contribution

It presents a novel approach that unifies tree search and active inference by framing planning as a structure learning problem with two new algorithms.

Findings

Forward propagation aligns with active inference.

Backward propagation relates to sophisticated inference.

The framework clarifies differences between planning strategies.

Abstract

Over the last 10 to 15 years, active inference has helped to explain various brain mechanisms from habit formation to dopaminergic discharge and even modelling curiosity. However, the current implementations suffer from an exponential (space and time) complexity class when computing the prior over all the possible policies up to the time-horizon. Fountas et al (2020) used Monte Carlo tree search to address this problem, leading to impressive results in two different tasks. In this paper, we present an alternative framework that aims to unify tree search and active inference by casting planning as a structure learning problem. Two tree search algorithms are then presented. The first propagates the expected free energy forward in time (i.e., towards the leaves), while the second propagates it backward (i.e., towards the root). Then, we demonstrate that forward and backward propagations are related to active inference and sophisticated inference, respectively, thereby clarifying the differences between those two planning strategies.