Free-Energy Minimizing Policies Under Generative Model Ambiguity

TL;DR

This paper introduces a variational free-energy approach for robust decision-making under model ambiguity, formulating a minimax control problem and providing an algorithm with convergence guarantees, demonstrated on a pendulum task.

Contribution

It develops a novel variational free-energy framework for distributionally robust control under model ambiguity, including an algorithm with proven convergence.

Findings

Proposed a variational free-energy formulation for robust control.

Proved that optimal policies require solving a non-convex minimization.

Demonstrated effectiveness through simulations on a pendulum swing-up.

Abstract

We present a variational free-energy formulation for distributionally robust decision-making with ambiguity in the generative model. The formulation, related to a broad range of learning and control frameworks, yields a minimax optimal control problem where maximization is over an uncertainty set that represents ambiguities. We prove that computing the optimal policy requires solving a non-convex minimization problem and propose an algorithm with convergence guarantees to find the solution. The effectiveness of our results is illustrated via simulations on a pendulum swing-up problem.