Statistical Analysis of Wasserstein Distributionally Robust Estimators

TL;DR

This paper analyzes Wasserstein distributionally robust estimators, providing a systematic method for selecting the adversary's budget, deriving error bounds, and establishing a central limit theorem for uncertainty quantification.

Contribution

It introduces a generic recipe for choosing the adversary's budget in Wasserstein DRO, linking it to confidence regions and error bounds that are dimension-free.

Findings

A finite-dimensional dual reformulation of the DRO problem.

A systematic prescription for selecting the adversary's budget.

A central limit theorem for the DRO estimator.

Abstract

We consider statistical methods which invoke a min-max distributionally robust formulation to extract good out-of-sample performance in data-driven optimization and learning problems. Acknowledging the distributional uncertainty in learning from limited samples, the min-max formulations introduce an adversarial inner player to explore unseen covariate data. The resulting Distributionally Robust Optimization (DRO) formulations, which include Wasserstein DRO formulations (our main focus), are specified using optimal transportation phenomena. Upon describing how these infinite-dimensional min-max problems can be approached via a finite-dimensional dual reformulation, the tutorial moves into its main component, namely, explaining a generic recipe for optimally selecting the size of the adversary's budget. This is achieved by studying the limit behavior of an optimal transport projection formulation arising from an inquiry on the smallest confidence region that includes the unknown population risk minimizer. Incidentally, this systematic prescription coincides with those in specific examples in high-dimensional statistics and results in error bounds that are free from the curse of dimensions. Equipped with this prescription, we present a central limit theorem for the DRO estimator and provide a recipe for constructing compatible confidence regions that are useful for uncertainty quantification. The rest of the tutorial is devoted to insights into the nature of the optimizers selected by the min-max formulations and additional applications of optimal transport projections.