Noise in Classification

TL;DR

This paper explores the challenges of learning linear thresholds in noisy environments, highlighting the difficulty introduced by adversarial noise and discussing strategies to overcome these issues through assumptions on data.

Contribution

It analyzes the impact of noise on learning linear thresholds and reviews methods to mitigate these challenges by leveraging natural data assumptions.

Findings

Noise significantly complicates learning linear thresholds.

Certain assumptions on data can improve learning robustness.

Adversarial noise renders some learning tasks computationally hard.

Abstract

This chapter considers the computational and statistical aspects of learning linear thresholds in presence of noise. When there is no noise, several algorithms exist that efficiently learn near-optimal linear thresholds using a small amount of data. However, even a small amount of adversarial noise makes this problem notoriously hard in the worst-case. We discuss approaches for dealing with these negative results by exploiting natural assumptions on the data-generating process.