# Wasserstein Distributionally Robust Stochastic Control: A Data-Driven Approach

## TL;DR

This paper develops a data-driven approach for designing control policies that are robust against distribution errors, using Wasserstein metrics and dynamic programming, with theoretical guarantees and explicit solutions for linear-quadratic cases.

## Contribution

It introduces computational algorithms for Wasserstein distributionally robust control, extending performance guarantees from single-stage to multi-stage problems without loss of confidence.

## Key findings

- Proposes tractable value and policy iteration algorithms.
- Provides explicit forms for optimal policies in linear-quadratic problems.
- Establishes out-of-sample performance guarantees using measure concentration.

## Abstract

Standard stochastic control methods assume that the probability distribution of uncertain variables is available. Unfortunately, in practice, obtaining accurate distribution information is a challenging task. To resolve this issue, we investigate the problem of designing a control policy that is robust against errors in the empirical distribution obtained from data. This problem can be formulated as a two-player zero-sum dynamic game problem, where the action space of the adversarial player is a Wasserstein ball centered at the empirical distribution. We propose computationally tractable value and policy iteration algorithms with explicit estimates of the number of iterations required for constructing an $\epsilon$-optimal policy. We show that the contraction property of associated Bellman operators extends a single-stage out-of-sample performance guarantee, obtained using a measure concentration inequality, to the corresponding multi-stage guarantee without any degradation in the confidence level. In addition, we characterize an explicit form of the optimal distributionally robust control policy and the worst-case distribution policy for linear-quadratic problems with Wasserstein penalty. Our study indicates that dynamic programming and Kantorovich duality play a critical role in solving and analyzing the Wasserstein distributionally robust stochastic control problems.