A dynamic programming principle for multiperiod control problems with bicausal constraints

TL;DR

This paper introduces a new metric on probability measures for multiperiod stochastic control problems, enabling a dynamic programming approach for distributionally robust optimization under bicausal constraints.

Contribution

The paper develops the adapted $(p, abla)$--Wasserstein distance, ensuring continuity and computability of control problems and DRO via dynamic programming, even with non-convex, non-compact uncertainty sets.

Findings

The adapted Wasserstein distance guarantees continuity of stochastic control problems.

Dynamic programming can be applied to Wasserstein-DRO problems using the new metric.

A minimax theorem holds for semi-separable cost functions despite non-convexity of uncertainty sets.

Abstract

We consider multiperiod stochastic control problems with non-parametric uncertainty on the underlying probabilistic model. We derive a new metric on the space of probability measures, called the adapted $(p, \infty)$--Wasserstein distance $\mathcal{AW}_p^\infty$ with the following properties: (1) the adapted $(p, \infty)$--Wasserstein distance generates a topology that guarantees continuity of stochastic control problems and (2) the corresponding $\mathcal{AW}_p^\infty$-distributionally robust optimization (DRO) problem can be computed via a dynamic programming principle involving one-step Wasserstein-DRO problems. If the cost function is semi-separable, then we further show that a minimax theorem holds, even though balls with respect to $\mathcal{AW}_p^\infty$ are neither convex nor compact in general. We also derive first-order sensitivity results.