Wasserstein Distributionally Robust Nonparametric Regression

TL;DR

This paper develops a nonparametric regression framework using Wasserstein distributionally robust optimization, establishing theoretical properties, error bounds, and demonstrating robustness and optimal convergence rates in high-dimensional settings.

Contribution

It introduces a novel nonparametric WDRO approach, characterizes the regularization effects based on Wasserstein order, and provides non-asymptotic error bounds with minimax optimal rates.

Findings

The order $k=1$ induces Lipschitz regularization, while $k>1$ relates to gradient-norm regularization.

The estimator achieves a convergence rate of $n^{-2eta/(d+2eta)}$, which is minimax optimal.

Simulation and MNIST experiments demonstrate the robustness of the proposed method.

Abstract

Wasserstein distributionally robust optimization (WDRO) strengthens statistical learning under model uncertainty by minimizing the local worst-case risk within a prescribed ambiguity set. Although WDRO has been extensively studied in parametric settings, its theoretical properties in nonparametric frameworks remain underexplored. This paper investigates WDRO for nonparametric regression. We first establish a structural distinction based on the order $k$ of the Wasserstein distance, showing that $k=1$ induces Lipschitz-type regularization, whereas $k > 1$ corresponds to gradient-norm regularization. To address model misspecification, we analyze the excess local worst-case risk, deriving non-asymptotic error bounds for estimators constructed using norm-constrained feedforward neural networks. This analysis is supported by new covering number and approximation bounds that simultaneously control both the function and its gradient. The proposed estimator achieves a convergence rate of $n^{-2\beta/(d+2\beta)}$ up to logarithmic factors, where $\beta$ depends on the target's smoothness and network parameters. This rate is shown to be minimax optimal under conditions commonly satisfied in high-dimensional settings. Moreover, these bounds on the excess local worst-case risk imply guarantees on the excess natural risk, ensuring robustness against any distribution within the ambiguity set. We show the framework's generality across regression and classification problems. Simulation studies and an application to the MNIST dataset further illustrate the estimator's robustness.