Non-Asymptotic Bounds for Adversarial Excess Risk under Misspecified Models

TL;DR

This paper develops non-asymptotic bounds for adversarial excess risk in misspecified models, providing theoretical guarantees for robust estimators under adversarial attacks in classification and regression.

Contribution

It introduces a unified framework for analyzing adversarial excess risk with non-asymptotic bounds, applicable to various loss functions and models.

Findings

Established non-asymptotic upper bounds for Lipschitz loss functions.

Applied results to classification and regression, improving bounds for quadratic loss.

Provided insights into the generalization and approximation errors in adversarial settings.

Abstract

We propose a general approach to evaluating the performance of robust estimators based on adversarial losses under misspecified models. We first show that adversarial risk is equivalent to the risk induced by a distributional adversarial attack under certain smoothness conditions. This ensures that the adversarial training procedure is well-defined. To evaluate the generalization performance of the adversarial estimator, we study the adversarial excess risk. Our proposed analysis method includes investigations on both generalization error and approximation error. We then establish non-asymptotic upper bounds for the adversarial excess risk associated with Lipschitz loss functions. In addition, we apply our general results to adversarial training for classification and regression problems. For the quadratic loss in nonparametric regression, we show that the adversarial excess risk bound can be improved over those for a general loss.