Distributional Active Inference

TL;DR

This paper introduces a formal framework that integrates active inference with distributional reinforcement learning, enhancing sample efficiency and removing the need for explicit transition models in complex control tasks.

Contribution

It provides a unified abstraction of reinforcement learning that incorporates active inference, bridging the gap between biological insights and artificial intelligence methods.

Findings

Seamless integration of active inference into distributional RL.

Enhanced sample efficiency without explicit transition models.

Unified framework applicable to model-based, distributional, and model-free approaches.

Abstract

Optimal control of complex environments with robotic systems faces two complementary and intertwined challenges: efficient organization of sensory state information and far-sighted action planning. Because the reinforcement learning framework addresses only the latter, it tends to deliver sample-inefficient solutions. Active inference is the state-of-the-art process theory that explains how biological brains handle this dual problem. However, its applications to artificial intelligence have thus far been limited to extensions of existing model-based approaches. We present a formal abstraction of reinforcement learning algorithms that spans model-based, distributional, and model-free approaches. This abstraction seamlessly integrates active inference into the distributional reinforcement learning framework, making its performance advantages accessible without transition dynamics modeling.