Reframing the Expected Free Energy: Four Formulations and a Unification

TL;DR

This paper formalizes the derivation of various formulations of expected free energy in active inference, unifying them under a single framework and analyzing two different settings with distinct assumptions and justifications.

Contribution

It introduces a unification approach for deriving multiple expected free energy formulations from a single root definition, clarifying their theoretical foundations.

Findings

All formulations can be derived from a single root expected free energy in one setting.

Limited prior preferences are compatible with the likelihood mapping in the first setting.

The second setting justifies only two formulations, focusing on risk and ambiguity.

Abstract

Active inference is a leading theory of perception, learning and decision making, which can be applied to neuroscience, robotics, psychology, and machine learning. Active inference is based on the expected free energy, which is mostly justified by the intuitive plausibility of its formulations, e.g., the risk plus ambiguity and information gain / pragmatic value formulations. This paper seek to formalize the problem of deriving these formulations from a single root expected free energy definition, i.e., the unification problem. Then, we study two settings, each one having its own root expected free energy definition. In the first setting, no justification for the expected free energy has been proposed to date, but all the formulations can be recovered from it. However, in this setting, the agent cannot have arbitrary prior preferences over observations. Indeed, only a limited class of prior preferences over observations is compatible with the likelihood mapping of the generative model. In the second setting, a justification of the root expected free energy definition is known, but this setting only accounts for two formulations, i.e., the risk over states plus ambiguity and entropy plus expected energy formulations.