Adversarial learning for nonparametric regression: Minimax rate and adaptive estimation

TL;DR

This paper investigates the statistical limits of adversarially robust nonparametric regression, establishing minimax rates and proposing adaptive estimators that achieve near-optimal performance under adversarial input perturbations.

Contribution

It derives the minimax convergence rate for adversarial nonparametric regression and introduces an adaptive estimator that nearly attains this rate across various classes.

Findings

Established the minimax rate under adversarial $L_q$-risks.

Proposed a piecewise local polynomial estimator achieving minimax optimality.

Designed an adaptive estimator that nearly matches the optimal rate across multiple classes.

Abstract

Despite tremendous advancements of machine learning models and algorithms in various application domains, they are known to be vulnerable to subtle, natural or intentionally crafted perturbations in future input data, known as adversarial attacks. While numerous adversarial learning methods have been proposed, fundamental questions about their statistical optimality in robust loss remain largely unanswered. In particular, the minimax rate of convergence and the construction of rate-optimal estimators under future $X$-attacks are yet to be worked out. In this paper, we address this issue in the context of nonparametric regression, under suitable assumptions on the smoothness of the regression function and the geometric structure of the input perturbation set. We first establish the minimax rate of convergence under adversarial $L_q$-risks with $1 \leq q \leq \infty$ and propose a piecewise local polynomial estimator that achieves the minimax optimality. The established minimax rate elucidates how the smoothness level and perturbation magnitude affect the fundamental limit of adversarial learning under future $X$-attacks. Furthermore, we construct a data-driven adaptive estimator that is shown to achieve, within a logarithmic factor, the optimal rate across a broad scale of nonparametric and adversarial classes.