Information-Theoretic Bounded Rationality

TL;DR

This paper presents an information-theoretic framework for bounded rationality, unifying decision-making under resource constraints with properties like controlling solution space size and enabling efficient Monte Carlo planning.

Contribution

It introduces a free energy functional as the core objective for bounded rational decisions, extending to sequential problems and unifying various classical decision rules.

Findings

The free energy functional controls the size of the solution space.

Monte Carlo planners can be exact and avoid exhaustive search.

The framework captures model uncertainty and generalizes classical decision rules.

Abstract

Bounded rationality, that is, decision-making and planning under resource limitations, is widely regarded as an important open problem in artificial intelligence, reinforcement learning, computational neuroscience and economics. This paper offers a consolidated presentation of a theory of bounded rationality based on information-theoretic ideas. We provide a conceptual justification for using the free energy functional as the objective function for characterizing bounded-rational decisions. This functional possesses three crucial properties: it controls the size of the solution space; it has Monte Carlo planners that are exact, yet bypass the need for exhaustive search; and it captures model uncertainty arising from lack of evidence or from interacting with other agents having unknown intentions. We discuss the single-step decision-making case, and show how to extend it to sequential decisions using equivalence transformations. This extension yields a very general class of decision problems that encompass classical decision rules (e.g. EXPECTIMAX and MINIMAX) as limit cases, as well as trust- and risk-sensitive planning.