Efficient search of active inference policy spaces using k-means

TL;DR

This paper introduces an efficient method for searching large policy spaces in active inference by embedding policies into a vector space and using k-means clustering to identify promising candidates, demonstrated on a graph-traversal task.

Contribution

The paper proposes a novel approach combining policy embeddings and k-means clustering to improve policy search efficiency in active inference.

Findings

Effective policy search in large spaces

Reduced computational complexity

Successful application to graph traversal problem

Abstract

We develop an approach to policy selection in active inference that allows us to efficiently search large policy spaces by mapping each policy to its embedding in a vector space. We sample the expected free energy of representative points in the space, then perform a more thorough policy search around the most promising point in this initial sample. We consider various approaches to creating the policy embedding space, and propose using k-means clustering to select representative points. We apply our technique to a goal-oriented graph-traversal problem, for which naive policy selection is intractable for even moderately large graphs.