Testing Noise Assumptions of Learning Algorithms

TL;DR

This paper introduces the first efficient method to test whether training data satisfies specific noise assumptions, enabling more reliable learning in noisy environments.

Contribution

It extends the testable learning framework to efficiently verify noise models and provides a polynomial-time algorithm for learning halfspaces with Massart noise.

Findings

Efficient algorithm for testing noise assumptions on training data.

Separation shown: testable learning is super-polynomial for random classification noise.

Polynomial-time testable learning algorithm for halfspaces with Massart noise.

Abstract

We pose a fundamental question in computational learning theory: can we efficiently test whether a training set satisfies the assumptions of a given noise model? This question has remained unaddressed despite decades of research on learning in the presence of noise. In this work, we show that this task is tractable and present the first efficient algorithm to test various noise assumptions on the training data. To model this question, we extend the recently proposed testable learning framework of Rubinfeld and Vasilyan (2023) and require a learner to run an associated test that satisfies the following two conditions: (1) whenever the test accepts, the learner outputs a classifier along with a certificate of optimality, and (2) the test must pass for any dataset drawn according to a specified modeling assumption on both the marginal distribution and the noise model. We then consider the problem of learning halfspaces over Gaussian marginals with Massart noise (where each label can be flipped with probability less than $1/2$ depending on the input features), and give a fully-polynomial time testable learning algorithm. We also show a separation between the classical setting of learning in the presence of structured noise and testable learning. In fact, for the simple case of random classification noise (where each label is flipped with fixed probability $\eta = 1/2$), we show that testable learning requires super-polynomial time while classical learning is trivial.