Learning Models with Uniform Performance via Distributionally Robust Optimization

TL;DR

This paper introduces a distributionally robust optimization framework that enhances model performance under distributional shifts, with theoretical guarantees and practical benefits demonstrated on real-world tasks.

Contribution

It develops a convex DRO formulation with convergence guarantees, finite-sample bounds, and limit theorems, advancing robust model learning under distributional uncertainty.

Findings

Improved generalization to unknown subpopulations

Enhanced tail performance in real tasks

Theoretical bounds and confidence intervals for robust models

Abstract

A common goal in statistics and machine learning is to learn models that can perform well against distributional shifts, such as latent heterogeneous subpopulations, unknown covariate shifts, or unmodeled temporal effects. We develop and analyze a distributionally robust stochastic optimization (DRO) framework that learns a model providing good performance against perturbations to the data-generating distribution. We give a convex formulation for the problem, providing several convergence guarantees. We prove finite-sample minimax upper and lower bounds, showing that distributional robustness sometimes comes at a cost in convergence rates. We give limit theorems for the learned parameters, where we fully specify the limiting distribution so that confidence intervals can be computed. On real tasks including generalizing to unknown subpopulations, fine-grained recognition, and providing good tail performance, the distributionally robust approach often exhibits improved performance.