# Deep Active Inference as Variational Policy Gradients

## TL;DR

This paper introduces a scalable deep Active Inference algorithm using neural networks, enabling application to complex tasks and showing performance comparable to reinforcement learning methods on benchmark environments.

## Contribution

It presents a novel deep Active Inference method that approximates densities with neural networks, allowing for larger, more complex task handling and bridging connections with reinforcement learning.

## Key findings

- Achieved competitive performance on OpenAIGym benchmarks.
- Demonstrated scalability of Active Inference with neural network approximations.
- Revealed connections between Active Inference, maximum entropy RL, and policy gradients.

## Abstract

Active Inference is a theory of action arising from neuroscience which casts action and planning as a bayesian inference problem to be solved by minimizing a single quantity - the variational free energy. Active Inference promises a unifying account of action and perception coupled with a biologically plausible process theory. Despite these potential advantages, current implementations of Active Inference can only handle small, discrete policy and state-spaces and typically require the environmental dynamics to be known. In this paper we propose a novel deep Active Inference algorithm which approximates key densities using deep neural networks as flexible function approximators, which enables Active Inference to scale to significantly larger and more complex tasks. We demonstrate our approach on a suite of OpenAIGym benchmark tasks and obtain performance comparable with common reinforcement learning baselines. Moreover, our algorithm shows similarities with maximum entropy reinforcement learning and the policy gradients algorithm, which reveals interesting connections between the Active Inference framework and reinforcement learning.