On the Value Function of Convex Bolza Problems Governed by Stochastic Difference Equations

TL;DR

This paper investigates the structure of the value function in stochastic convex Bolza problems, establishing a stochastic Hamiltonian difference equation for subgradients and exploring duality and minimizer existence.

Contribution

It extends the method of characteristics to stochastic discrete-time problems and analyzes the subdifferential structure of the value function using duality and minimizer existence.

Findings

Connected subgradients evolution with stochastic Hamiltonian difference equations

Developed dual representation for the value function in stochastic setting

Studied minimizer existence for linear stochastic control problems

Abstract

In this paper we study the value function of Bolza problems governed by stochastic difference equations, with particular emphasis on the convex non-anticipative case. Our goal is to provide some insights on the structure of the subdiferential of the value function. In particular, we establish a connection between the evolution of the subgradients of the value function and a stochastic difference equation of Hamiltonian type. This result can be seen as a transposition of the method of characteristics, introduced by Rockafellar and Wolenski in the 2000s, to the stochastic discrete-time setting. Similarly as done in the literature for the deterministic case, the analysis is based on a duality approach. For this reason we study first a dual representation for the value function in terms of the value function of a dual problem, which is a pseudo Bolza problem. The main difference with the deterministic case is that (due to the non-anticipativity) the symmetry between the Bolza problem and its dual is no longer valid. This in turn implies that ensuring the existence of minimizers for the Bolza problem (which is a key point for establishing the method of characteristics) is not as simple as in the deterministic case, and it should be addressed differently. To complete the exposition, we study the existence of minimizers for a particular class of Bolza problems governed by linear stochastic difference equations, the so-called linear-convex optimal control problems.