Bayesian policy selection using active inference

TL;DR

This paper introduces a Bayesian policy selection method based on active inference and the free energy principle, demonstrating its application to the mountain car problem and its ability to unify reinforcement learning and learning from demonstrations.

Contribution

It applies active inference to policy selection, providing a normative, biologically inspired framework that addresses RL challenges like reward shaping and sample inefficiency.

Findings

Successfully applied to the mountain car problem

Unifies reinforcement learning and learning from demonstrations

Addresses issues of reward shaping and generalization

Abstract

Learning to take actions based on observations is a core requirement for artificial agents to be able to be successful and robust at their task. Reinforcement Learning (RL) is a well-known technique for learning such policies. However, current RL algorithms often have to deal with reward shaping, have difficulties generalizing to other environments and are most often sample inefficient. In this paper, we explore active inference and the free energy principle, a normative theory from neuroscience that explains how self-organizing biological systems operate by maintaining a model of the world and casting action selection as an inference problem. We apply this concept to a typical problem known to the RL community, the mountain car problem, and show how active inference encompasses both RL and learning from demonstrations.