A Simulation Study on Turnpikes in Stochastic LQ Optimal Control

TL;DR

This study uses Polynomial Chaos Expansions to explore the presence and characteristics of turnpike phenomena in stochastic linear-quadratic optimal control problems, revealing their robustness under variance penalization.

Contribution

It introduces a novel simulation approach employing PCE to analyze turnpikes in stochastic LQ problems, highlighting their persistence under different objective modifications.

Findings

Turnpikes are observable in PCE coefficients and statistical moments.

Turnpike phenomena occur in optimal trajectories and stationary distributions.

Variance penalization modifies but does not eliminate turnpikes.

Abstract

This paper presents a simulation study on turnpike phenomena in stochastic optimal control problems. We employ the framework of Polynomial Chaos Expansions (PCE) to investigate the presence of turnpikes in stochastic LQ problems. Our findings indicate that turnpikes can be observed in the evolution of PCE coefficients as well as in the evolution of statistical moments. Moreover, the turnpike phenomenon can be observed for optimal realization trajectories and with respect to the optimal stationary distribution. Finally, while adding variance penalization to the objective alters the turnpike, it does not destroy the phenomenon.