Efficient PAC Learning of Halfspaces with Constant Malicious Noise Rate

TL;DR

This paper presents an efficient method for PAC learning of halfspaces that tolerates a constant rate of malicious noise, combining distributional and margin conditions with a reweighted hinge loss approach.

Contribution

It introduces a novel algorithm and analysis that enable constant noise tolerance in learning halfspaces under specific distributional and margin assumptions.

Findings

Achieves constant malicious noise tolerance in PAC learning of halfspaces.

Develops a new weighted hinge loss analysis for robustness against adversarial corruption.

Provides an efficient algorithm controlling gradient deterioration from corrupted samples.

Abstract

Understanding noise tolerance of machine learning algorithms is a central quest in learning theory. In this work, we study the problem of computationally efficient PAC learning of halfspaces in the presence of malicious noise, where an adversary can corrupt both instances and labels of training samples. The best-known noise tolerance either depends on a target error rate under distributional assumptions or on a margin parameter under large-margin conditions. In this work, we show that when both types of conditions are satisfied, it is possible to achieve constant noise tolerance by minimizing a reweighted hinge loss. Our key ingredients include: 1) an efficient algorithm that finds weights to control the gradient deterioration from corrupted samples, and 2) a new analysis on the robustness of the hinge loss equipped with such weights.