Rationally inattentive control of Markov processes

TL;DR

This paper develops a convex optimization framework for optimal control of Markov processes under information constraints, extending classical control theory with mutual information limits and illustrating it with LQG control examples.

Contribution

It introduces a novel convex formulation of average-cost optimal control with information constraints, linking it to approximate dynamic programming and Bellman error decomposition.

Findings

The model reduces to a linear programming problem with mutual information constraints.

The approach is applicable to linear-quadratic-Gaussian control problems.

Some results are extended to infinite-horizon discounted costs.

Abstract

The article poses a general model for optimal control subject to information constraints, motivated in part by recent work of Sims and others on information-constrained decision-making by economic agents. In the average-cost optimal control framework, the general model introduced in this paper reduces to a variant of the linear-programming representation of the average-cost optimal control problem, subject to an additional mutual information constraint on the randomized stationary policy. The resulting optimization problem is convex and admits a decomposition based on the Bellman error, which is the object of study in approximate dynamic programming. The theory is illustrated through the example of information-constrained linear-quadratic-Gaussian (LQG) control problem. Some results on the infinite-horizon discounted-cost criterion are also presented.