Binary Classification Under $\ell_0$ Attacks for General Noise Distribution

TL;DR

This paper investigates binary classification under $\, ext{ extlbrackdbl}\, ext{ extbrackdbl}\,0$-norm adversarial attacks, proposing a method that neutralizes the adversary's effect when perturbations are limited to $\, extsqrt{d}$ samples, revealing a phase transition at this threshold.

Contribution

The paper introduces a novel classification method with truncation to counter $\, ext{ extlbrackdbl}\, ext{ extbrackdbl}\,0$-norm attacks and characterizes the phase transition in adversarial perturbation limits.

Findings

Almost optimal classification error when adversary perturbs up to $\, extsqrt{d}$ samples.

Complete neutralization of adversarial effect below the $\, extsqrt{d}$ threshold.

No classifier can outperform random guessing if perturbations exceed $\, extsqrt{d}$ samples.

Abstract

Adversarial examples have recently drawn considerable attention in the field of machine learning due to the fact that small perturbations in the data can result in major performance degradation. This phenomenon is usually modeled by a malicious adversary that can apply perturbations to the data in a constrained fashion, such as being bounded in a certain norm. In this paper, we study this problem when the adversary is constrained by the $\ell_0$ norm; i.e., it can perturb a certain number of coordinates in the input, but has no limit on how much it can perturb those coordinates. Due to the combinatorial nature of this setting, we need to go beyond the standard techniques in robust machine learning to address this problem. We consider a binary classification scenario where $d$ noisy data samples of the true label are provided to us after adversarial perturbations. We introduce a classification method which employs a nonlinear component called truncation, and show in an asymptotic scenario, as long as the adversary is restricted to perturb no more than $\sqrt{d}$ data samples, we can almost achieve the optimal classification error in the absence of the adversary, i.e. we can completely neutralize adversary's effect. Surprisingly, we observe a phase transition in the sense that using a converse argument, we show that if the adversary can perturb more than $\sqrt{d}$ coordinates, no classifier can do better than a random guess.