On Rate-Distortion Theory in Capacity-Limited Cognition & Reinforcement Learning

TL;DR

This paper surveys information-theoretic models of capacity-limited decision making in biological and artificial agents, emphasizing the role of rate-distortion theory in understanding bounded rationality and efficient learning.

Contribution

It provides a concise overview of how rate-distortion theory formalizes capacity constraints in decision-making and learning, connecting cognitive science and reinforcement learning.

Findings

Highlights the use of rate-distortion theory in modeling bounded rationality.

Connects information-theoretic principles with Bayesian regret bounds.

Summarizes key models in biological and artificial decision-making.

Abstract

Throughout the cognitive-science literature, there is widespread agreement that decision-making agents operating in the real world do so under limited information-processing capabilities and without access to unbounded cognitive or computational resources. Prior work has drawn inspiration from this fact and leveraged an information-theoretic model of such behaviors or policies as communication channels operating under a bounded rate constraint. Meanwhile, a parallel line of work also capitalizes on the same principles from rate-distortion theory to formalize capacity-limited decision making through the notion of a learning target, which facilitates Bayesian regret bounds for provably-efficient learning algorithms. In this paper, we aim to elucidate this latter perspective by presenting a brief survey of these information-theoretic models of capacity-limited decision making in biological and artificial agents.