Minimax rates of convergence for nonparametric regression under adversarial attacks

TL;DR

This paper theoretically analyzes the limits of robustness in nonparametric regression under adversarial input perturbations, establishing minimax convergence rates and optimal procedures for such adversarial settings.

Contribution

It derives the minimax rates of convergence under adversarial attacks in nonparametric regression and proposes an optimal plug-in estimator for robustness.

Findings

Minimax rate under adversarial attack equals the sum of standard rate and maximum deviation within the function class.

Optimal rates can be achieved by an adversarial plug-in procedure based on a standard minimax estimator.

Illustrative examples demonstrate the theoretical minimax results.

Abstract

Recent research shows the susceptibility of machine learning models to adversarial attacks, wherein minor but maliciously chosen perturbations of the input can significantly degrade model performance. In this paper, we theoretically analyse the limits of robustness against such adversarial attacks in a nonparametric regression setting, by examining the minimax rates of convergence in an adversarial sup-norm. Our work reveals that the minimax rate under adversarial attacks in the input is the same as sum of two terms: one represents the minimax rate in the standard setting without adversarial attacks, and the other reflects the maximum deviation of the true regression function value within the target function class when subjected to the input perturbations. The optimal rates under the adversarial setup can be achieved by an adversarial plug-in procedure constructed from a minimax optimal estimator in the corresponding standard setting. Two specific examples are given to illustrate the established minimax results.