Deep Active Inference for Partially Observable MDPs

TL;DR

This paper introduces a deep active inference model capable of learning effective policies directly from high-dimensional sensory inputs in partially observable environments, outperforming some existing reinforcement learning methods.

Contribution

The paper presents a novel deep active inference framework that handles partial observability using variational autoencoders, extending previous models limited to fully observable domains.

Findings

Achieves comparable or superior performance to deep Q-learning on OpenAI benchmarks.

Successfully learns policies directly from high-dimensional sensory data.

Demonstrates the applicability of active inference in complex, partially observable settings.

Abstract

Deep active inference has been proposed as a scalable approach to perception and action that deals with large policy and state spaces. However, current models are limited to fully observable domains. In this paper, we describe a deep active inference model that can learn successful policies directly from high-dimensional sensory inputs. The deep learning architecture optimizes a variant of the expected free energy and encodes the continuous state representation by means of a variational autoencoder. We show, in the OpenAI benchmark, that our approach has comparable or better performance than deep Q-learning, a state-of-the-art deep reinforcement learning algorithm.