Central Limit Theorems for Sample Average Approximations in Stochastic Optimal Control

TL;DR

This paper proves central limit theorems for the Sample Average Approximation method in stochastic optimal control, revealing how statistical uncertainty propagates through dynamic programming and providing explicit variance formulas.

Contribution

It introduces a recursive framework for Gaussian limits of SAA value functions in stochastic control, including explicit variance decompositions and applications to LQR.

Findings

Gaussian limits for SAA value functions under unique policies

Variance decomposition into current and future stages

Explicit variance formulas for linear quadratic regulator

Abstract

We establish central limit theorems for the Sample Average Approximation (SAA) method in discrete-time, finite-horizon stochastic optimal control. Our analysis is based on an abstract limit theorem for stochastic backward recursions, which yields a recursive characterization of the limiting laws. Applied to the dynamic programming principle, this framework gives Gaussian limits for SAA value functions under unique optimal policies. The asymptotic variance at each stage decomposes into a current-stage variance and a propagated future variance, demonstrating how statistical uncertainty accumulates backward through time. We also apply the framework to the linear quadratic regulator, derive explicit limiting laws and variance formulas, and provide numerical illustrations of the resulting variance decomposition. Finally, we discuss the form of the limit laws under nonunique optimal policies.