Attribute noise robust binary classification

TL;DR

This paper investigates the robustness of different loss functions for binary classification with noisy features, showing squared loss's robustness in certain models and empirical support for its effectiveness.

Contribution

It introduces theoretical analysis of loss function robustness under attribute noise models and compares the practical performance of squared loss versus 0-1 loss.

Findings

Squared loss is robust under Sy-De attribute noise model.

0-1 loss is robust in 2D under Asy-In attribute noise but is computationally intractable.

Empirical results support squared loss robustness at low to moderate noise levels.

Abstract

We consider the problem of learning linear classifiers when both features and labels are binary. In addition, the features are noisy, i.e., they could be flipped with an unknown probability. In Sy-De attribute noise model, where all features could be noisy together with same probability, we show that $0$-$1$ loss ($l_{0-1}$) need not be robust but a popular surrogate, squared loss ($l_{sq}$) is. In Asy-In attribute noise model, we prove that $l_{0-1}$ is robust for any distribution over 2 dimensional feature space. However, due to computational intractability of $l_{0-1}$, we resort to $l_{sq}$ and observe that it need not be Asy-In noise robust. Our empirical results support Sy-De robustness of squared loss for low to moderate noise rates.