Bayesian Reinforcement Learning with Limited Cognitive Load

TL;DR

This paper reviews recent advances in capacity-limited Bayesian reinforcement learning, highlighting how processing constraints influence learning and decision-making, with implications for cognitive science.

Contribution

It synthesizes recent algorithms and theoretical results in capacity-limited Bayesian RL, bridging reinforcement learning, Bayesian decision-making, and rate-distortion theory.

Findings

Provides an accessible review of algorithms and theories

Highlights applications to cognitive and behavioral sciences

Connects capacity constraints with learning dynamics

Abstract

All biological and artificial agents must learn and make decisions given limits on their ability to process information. As such, a general theory of adaptive behavior should be able to account for the complex interactions between an agent's learning history, decisions, and capacity constraints. Recent work in computer science has begun to clarify the principles that shape these dynamics by bridging ideas from reinforcement learning, Bayesian decision-making, and rate-distortion theory. This body of work provides an account of capacity-limited Bayesian reinforcement learning, a unifying normative framework for modeling the effect of processing constraints on learning and action selection. Here, we provide an accessible review of recent algorithms and theoretical results in this setting, paying special attention to how these ideas can be applied to studying questions in the cognitive and behavioral sciences.