Contrastive Active Inference

TL;DR

This paper introduces a contrastive objective for active inference that improves scalability, reduces computational costs, and enhances performance in complex, image-based tasks, matching reinforcement learning in goal achievement without requiring explicit rewards.

Contribution

It proposes a novel contrastive approach for active inference that outperforms likelihood-based methods and matches reinforcement learning in complex environments.

Findings

Outperforms likelihood-based active inference in image tasks

Computationally cheaper and easier to train

Handles distractors and goal generalization effectively

Abstract

Active inference is a unifying theory for perception and action resting upon the idea that the brain maintains an internal model of the world by minimizing free energy. From a behavioral perspective, active inference agents can be seen as self-evidencing beings that act to fulfill their optimistic predictions, namely preferred outcomes or goals. In contrast, reinforcement learning requires human-designed rewards to accomplish any desired outcome. Although active inference could provide a more natural self-supervised objective for control, its applicability has been limited because of the shortcomings in scaling the approach to complex environments. In this work, we propose a contrastive objective for active inference that strongly reduces the computational burden in learning the agent's generative model and planning future actions. Our method performs notably better than likelihood-based active inference in image-based tasks, while also being computationally cheaper and easier to train. We compare to reinforcement learning agents that have access to human-designed reward functions, showing that our approach closely matches their performance. Finally, we also show that contrastive methods perform significantly better in the case of distractors in the environment and that our method is able to generalize goals to variations in the background. Website and code: https://contrastive-aif.github.io/