Smoothed $f$-Divergence Distributionally Robust Optimization

TL;DR

This paper introduces a novel distributionally robust optimization framework using smoothed $f$-divergences that improves out-of-sample performance bounds and maintains computational efficiency.

Contribution

It proposes a new DRO formulation with smoothed $f$-divergences that offers near-tight statistical bounds and computational tractability for data-driven optimization.

Findings

Achieves nearly tight exponential decay bounds on out-of-sample performance.

Maintains computational complexity comparable to existing DRO methods.

Provides a statistically optimal approach for various objective functions and distributions.

Abstract

In data-driven optimization, sample average approximation (SAA) is known to suffer from the so-called optimizer's curse that causes an over-optimistic evaluation of the solution performance. We argue that a special type of distributionallly robust optimization (DRO) formulation offers theoretical advantages in correcting for this optimizer's curse compared to simple ``margin'' adjustments to SAA and other DRO approaches: It attains a statistical bound on the out-of-sample performance, for a wide class of objective functions and distributions, that is nearly tightest in terms of exponential decay rate. This DRO uses an ambiguity set based on a Kullback Leibler (KL) divergence smoothed by the Wasserstein or L\'evy-Prokhorov (LP) distance via a suitable distance optimization. Computationally, we also show that such a DRO, and its generalized versions using smoothed $f$-divergence, are not harder than DRO problems based on $f$-divergence or Wasserstein distances, rendering our DRO formulations both statistically optimal and computationally viable.