Dynamic Programming for General Linear Quadratic Optimal Stochastic Control with Random Coefficients

TL;DR

This paper analyzes a stochastic linear-quadratic control problem with random coefficients, proving the quadratic form of the value function, deriving the associated backward stochastic Riccati equation, and establishing existence and uniqueness of solutions.

Contribution

It provides a comprehensive solution to the backward stochastic Riccati equation using dynamic programming, extending previous work to more general stochastic control systems.

Findings

Value function is quadratic in state variable

K process is a continuous semi-martingale solving BSRE

Existence and uniqueness of solutions to the BSRE are established

Abstract

We are concerned with the linear-quadratic optimal stochastic control problem with random coefficients. Under suitable conditions, we prove that the value field $V(t,x,\omega), (t,x,\omega)\in [0,T]\times R^n\times \Omega$, is quadratic in $x$, and has the following form: $V(t,x)=\langle K_tx, x\rangle$ where $K$ is an essentially bounded nonnegative symmetric matrix-valued adapted processes. Using the dynamic programming principle (DPP), we prove that $K$ is a continuous semi-martingale of the form $$K_t=K_0+\int_0^t \, dk_s+\sum_{i=1}^d\int_0^tL_s^i\, dW_s^i, \quad t\in [0,T]$$ with $k$ being a continuous process of bounded variation and $$E\left[\left(\int_0^T|L_s|^2\, ds\right)^p\right] <\infty, \quad \forall p\ge 2; $$ and that $(K, L)$ with $L:=(L^1, \cdots, L^d)$ is a solution to the associated backward stochastic Riccati equation (BSRE), whose generator is highly nonlinear in the unknown pair of processes. The uniqueness is also proved via a localized completion of squares in a self-contained manner for a general BSRE. The existence and uniqueness of adapted solution to a general BSRE was initially proposed by the French mathematician J. M. Bismut (1976, 1978). It had been solved by the author (2003) via the stochastic maximum principle with a viewpoint of stochastic flow for the associated stochastic Hamiltonian system. The present paper is its companion, and gives the {\it second but more comprehensive} adapted solution to a general BSRE via the DDP. Further extensions to the jump-diffusion control system and to the general nonlinear control system are possible.