On the Relationship Between Active Inference and Control as Inference

TL;DR

This paper compares Active Inference and Control-as-Inference frameworks, clarifying their relationship and differences in how they incorporate value, with implications for advancing decision-making models in brain sciences and reinforcement learning.

Contribution

It provides a formal comparison of AIF and CAI, revealing that their main difference lies in value integration within their generative models.

Findings

Identifies key differences in value incorporation between AIF and CAI.

Highlights potential for mutual insights between the frameworks.

Clarifies the relationship to guide future research in decision-making models.

Abstract

Active Inference (AIF) is an emerging framework in the brain sciences which suggests that biological agents act to minimise a variational bound on model evidence. Control-as-Inference (CAI) is a framework within reinforcement learning which casts decision making as a variational inference problem. While these frameworks both consider action selection through the lens of variational inference, their relationship remains unclear. Here, we provide a formal comparison between them and demonstrate that the primary difference arises from how value is incorporated into their respective generative models. In the context of this comparison, we highlight several ways in which these frameworks can inform one another.