The role of information in high dimensional stochastic optimal control

TL;DR

This paper studies the impact of partial observations on high-dimensional stochastic optimal control, revealing how fluctuations differ from the mean-field limit and proposing an approximate Kalman filter approach.

Contribution

It introduces a central limit theorem for Gaussian fluctuations in mean-field control under partial observations and demonstrates a practical filtering method.

Findings

Partial observations significantly influence fluctuations.

Kalman filter provides an effective approximation method.

Fluctuations diverge at a phase transition point.

Abstract

The stochastic optimal control of many agents is an important problem in various fields. We investigate the problem of partial observations, where the state of each agent is not fully observed and the control must be decided based on noisy observations. This results in a high-dimensional Markov decision process that is impractical to handle directly. However, in the limit as the number of agents approaches infinity, a finite-dimensional mean-field optimal control problem emerges, which coincides with the problem of full information. Our main contribution is to investigate a central limit theorem for the Gaussian fluctuations of the mean-field optimal control. Our findings show that partial observations play an essential role in the fluctuations, in contrast to the mean-field limit. We establish a method that uses an approximate Kalman filter, which is straightforward to compute even when the number of states is large. This provides some theoretical evidence of the efficacy of Kalman filter methods that are commonly used across a range of practical applications. We demonstrate our results with two examples: an epidemic model with observations of positive tests and a simple two-state model that exhibits a phase transition at which point the fluctuations diverge.