Active inference and artificial reasoning

TL;DR

This paper presents a method for active inference that optimizes information gain to improve structure learning and decision-making in artificial agents, using Bayesian model reduction and experimental design principles.

Contribution

It introduces a principled approach to active inference that combines information gain with model selection, enhancing sample efficiency in artificial reasoning.

Findings

Efficient selection of informative outcomes improves model disambiguation.

Bayesian Model Reduction enables quick evaluation of model posteriors.

Application to discrete models demonstrates enhanced reasoning capabilities.

Abstract

This technical note considers the sampling of outcomes that provide the greatest amount of information about the structure of underlying world models. This generalisation furnishes a principled approach to structure learning under a plausible set of generative models or hypotheses. In active inference, policies - i.e., combinations of actions - are selected based on their expected free energy, which comprises expected information gain and value. Information gain corresponds to the KL divergence between predictive posteriors with, and without, the consequences of action. Posteriors over models can be evaluated quickly and efficiently using Bayesian Model Reduction, based upon accumulated posterior beliefs about model parameters. The ensuing information gain can then be used to select actions that disambiguate among alternative models, in the spirit of optimal experimental design. We illustrate this kind of active selection or reasoning using partially observed discrete models; namely, a 'three-ball' paradigm used previously to describe artificial insight and 'aha moments' via (synthetic) introspection or sleep. We focus on the sample efficiency afforded by seeking outcomes that resolve the greatest uncertainty about the world model, under which outcomes are generated.