Robustness for Non-Parametric Classification: A Generic Attack and Defense

TL;DR

This paper introduces a universal attack and defense strategy for non-parametric classifiers like nearest neighbors and decision trees, providing theoretical insights and empirical validation to enhance robustness.

Contribution

It presents a general defense method called adversarial pruning, a novel attack applicable to various classifiers, and derives an optimally robust classifier analogous to Bayes Optimal.

Findings

Adversarial pruning improves robustness of non-parametric classifiers.

The novel attack effectively challenges existing defenses.

Empirical results show the proposed methods outperform or match prior approaches.

Abstract

Adversarially robust machine learning has received much recent attention. However, prior attacks and defenses for non-parametric classifiers have been developed in an ad-hoc or classifier-specific basis. In this work, we take a holistic look at adversarial examples for non-parametric classifiers, including nearest neighbors, decision trees, and random forests. We provide a general defense method, adversarial pruning, that works by preprocessing the dataset to become well-separated. To test our defense, we provide a novel attack that applies to a wide range of non-parametric classifiers. Theoretically, we derive an optimally robust classifier, which is analogous to the Bayes Optimal. We show that adversarial pruning can be viewed as a finite sample approximation to this optimal classifier. We empirically show that our defense and attack are either better than or competitive with prior work on non-parametric classifiers. Overall, our results provide a strong and broadly-applicable baseline for future work on robust non-parametrics. Code available at https://github.com/yangarbiter/adversarial-nonparametrics/ .