Robust Nonparametric Regression under Poisoning Attack

TL;DR

This paper develops a robust nonparametric regression method resilient to adversarial sample modifications, using an M-estimator with a correction step to achieve near minimax optimality under attack.

Contribution

It introduces a robust M-estimator for nonparametric regression under poisoning attacks and proposes a correction method to improve robustness for large attack sizes.

Findings

The M-estimator significantly reduces the impact of malicious samples.

The convergence rate and minimax bounds are established for the proposed method.

The correction step enhances robustness, achieving near minimax optimality for arbitrary attack sizes.

Abstract

This paper studies robust nonparametric regression, in which an adversarial attacker can modify the values of up to $q$ samples from a training dataset of size $N$. Our initial solution is an M-estimator based on Huber loss minimization. Compared with simple kernel regression, i.e. the Nadaraya-Watson estimator, this method can significantly weaken the impact of malicious samples on the regression performance. We provide the convergence rate as well as the corresponding minimax lower bound. The result shows that, with proper bandwidth selection, $\ell_\infty$ error is minimax optimal. The $\ell_2$ error is optimal with relatively small $q$, but is suboptimal with larger $q$. The reason is that this estimator is vulnerable if there are many attacked samples concentrating in a small region. To address this issue, we propose a correction method by projecting the initial estimate to the space of Lipschitz functions. The final estimate is nearly minimax optimal for arbitrary $q$, up to a $\ln N$ factor.