An Adversarial Interpretation of Information-Theoretic Bounded Rationality

TL;DR

This paper reveals that information-theoretic bounded rationality modeled through free energy optimization can be interpreted as a game against an adversary, linking decision-making under constraints to game theory and Nash equilibria.

Contribution

It establishes a novel equivalence between single-agent free energy optimization and a game against an adversary, clarifying the decision-making process under information constraints.

Findings

Single-agent free energy optimization is equivalent to an adversarial game.

The adversary's optimal strategy makes the decision maker indifferent among choices.

This connection tightens the relationship between bounded rationality and game theory.

Abstract

Recently, there has been a growing interest in modeling planning with information constraints. Accordingly, an agent maximizes a regularized expected utility known as the free energy, where the regularizer is given by the information divergence from a prior to a posterior policy. While this approach can be justified in various ways, including from statistical mechanics and information theory, it is still unclear how it relates to decision-making against adversarial environments. This connection has previously been suggested in work relating the free energy to risk-sensitive control and to extensive form games. Here, we show that a single-agent free energy optimization is equivalent to a game between the agent and an imaginary adversary. The adversary can, by paying an exponential penalty, generate costs that diminish the decision maker's payoffs. It turns out that the optimal strategy of the adversary consists in choosing costs so as to render the decision maker indifferent among its choices, which is a definining property of a Nash equilibrium, thus tightening the connection between free energy optimization and game theory.