Hedging Complexity in Generalization via a Parametric Distributionally Robust Optimization Framework

TL;DR

This paper introduces a parametric distributionally robust optimization framework that reduces complexity-related errors in generalization, especially under distribution shifts, with proven improved bounds and demonstrated effectiveness on portfolio and regression tasks.

Contribution

It proposes a novel parametric DRO approach that mitigates complexity issues in high-dimensional stochastic optimization, providing better generalization bounds and robustness.

Findings

Significantly improved generalization bounds over existing methods.

Effective under distribution shifts and in contextual optimization.

Demonstrated superior performance on synthetic and real-world tasks.

Abstract

Empirical risk minimization (ERM) and distributionally robust optimization (DRO) are popular approaches for solving stochastic optimization problems that appear in operations management and machine learning. Existing generalization error bounds for these methods depend on either the complexity of the cost function or dimension of the random perturbations. Consequently, the performance of these methods can be poor for high-dimensional problems with complex objective functions. We propose a simple approach in which the distribution of random perturbations is approximated using a parametric family of distributions. This mitigates both sources of complexity; however, it introduces a model misspecification error. We show that this new source of error can be controlled by suitable DRO formulations. Our proposed parametric DRO approach has significantly improved generalization bounds over existing ERM and DRO methods and parametric ERM for a wide variety of settings. Our method is particularly effective under distribution shifts and works broadly in contextual optimization. We also illustrate the superior performance of our approach on both synthetic and real-data portfolio optimization and regression tasks.