Distributionally Robust Optimization via Ball Oracle Acceleration

TL;DR

This paper introduces an accelerated algorithm for distributionally robust optimization that efficiently minimizes convex losses within uncertainty sets, improving query complexity over previous methods.

Contribution

It develops a novel accelerated method using a ball oracle for DRO, achieving better complexity bounds for non-smooth loss functions.

Findings

Achieves $ ilde{O}(N ext{ } ext{epsilon}^{-2/3} + ext{epsilon}^{-2})$ query complexity.

Improves existing algorithms' complexity by a factor of up to $ ext{epsilon}^{-4/3}$.

Provides efficient implementations of the ball oracle for DRO objectives.

Abstract

We develop and analyze algorithms for distributionally robust optimization (DRO) of convex losses. In particular, we consider group-structured and bounded $f$-divergence uncertainty sets. Our approach relies on an accelerated method that queries a ball optimization oracle, i.e., a subroutine that minimizes the objective within a small ball around the query point. Our main contribution is efficient implementations of this oracle for DRO objectives. For DRO with $N$ non-smooth loss functions, the resulting algorithms find an $\epsilon$-accurate solution with $\widetilde{O}\left(N\epsilon^{-2/3} + \epsilon^{-2}\right)$ first-order oracle queries to individual loss functions. Compared to existing algorithms for this problem, we improve complexity by a factor of up to $\epsilon^{-4/3}$.