Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihood

TL;DR

This paper introduces a nonparametric hypothesis testing method for class-conditional label noise using local maximum likelihood, offering more flexible models than traditional parametric approaches.

Contribution

It extends existing hypothesis testing techniques to nonparametric logistic regression, reducing model misspecification issues and broadening applicability.

Findings

The nonparametric approach performs well on synthetic data.

It demonstrates advantages over parametric methods in real-world case studies.

The method is less susceptible to model misspecification.

Abstract

In supervised learning, automatically assessing the quality of the labels before any learning takes place remains an open research question. In certain particular cases, hypothesis testing procedures have been proposed to assess whether a given instance-label dataset is contaminated with class-conditional label noise, as opposed to uniform label noise. The existing theory builds on the asymptotic properties of the Maximum Likelihood Estimate for parametric logistic regression. However, the parametric assumptions on top of which these approaches are constructed are often too strong and unrealistic in practice. To alleviate this problem, in this paper we propose an alternative path by showing how similar procedures can be followed when the underlying model is a product of Local Maximum Likelihood Estimation that leads to more flexible nonparametric logistic regression models, which in turn are less susceptible to model misspecification. This different view allows for wider applicability of the tests by offering users access to a richer model class. Similarly to existing works, we assume we have access to anchor points which are provided by the users. We introduce the necessary ingredients for the adaptation of the hypothesis tests to the case of nonparametric logistic regression and empirically compare against the parametric approach presenting both synthetic and real-world case studies and discussing the advantages and limitations of the proposed approach.