Analyzing the Robustness of Nearest Neighbors to Adversarial Examples

TL;DR

This paper develops a theoretical framework to analyze the robustness of nearest neighbor classifiers against adversarial examples, revealing how the choice of k influences robustness and proposing a modified classifier with guaranteed robustness.

Contribution

Introduces a bias-variance inspired framework for understanding adversarial robustness in nearest neighbors and proposes a new robust 1-NN classifier with theoretical guarantees.

Findings

Robustness depends critically on the value of k in k-NN.

Robustness approaches Bayes Optimal as k increases.

Modified 1-NN classifier guarantees robustness in large samples.

Abstract

Motivated by safety-critical applications, test-time attacks on classifiers via adversarial examples has recently received a great deal of attention. However, there is a general lack of understanding on why adversarial examples arise; whether they originate due to inherent properties of data or due to lack of training samples remains ill-understood. In this work, we introduce a theoretical framework analogous to bias-variance theory for understanding these effects. We use our framework to analyze the robustness of a canonical non-parametric classifier - the k-nearest neighbors. Our analysis shows that its robustness properties depend critically on the value of k - the classifier may be inherently non-robust for small k, but its robustness approaches that of the Bayes Optimal classifier for fast-growing k. We propose a novel modified 1-nearest neighbor classifier, and guarantee its robustness in the large sample limit. Our experiments suggest that this classifier may have good robustness properties even for reasonable data set sizes.