Interactive Learning from Multiple Noisy Labels

TL;DR

This paper introduces a novel interactive learning approach that leverages disagreements among multiple noisy labels to identify meaningful examples, improving model parameter estimation and understanding perceptron performance.

Contribution

It proposes a new method that uses annotator disagreement to select informative examples and analyzes perceptron performance within this framework.

Findings

Effective estimation of latent variable model parameters

Demonstrated usefulness on synthetic and benchmark datasets

Theoretical analysis of perceptron performance

Abstract

Interactive learning is a process in which a machine learning algorithm is provided with meaningful, well-chosen examples as opposed to randomly chosen examples typical in standard supervised learning. In this paper, we propose a new method for interactive learning from multiple noisy labels where we exploit the disagreement among annotators to quantify the easiness (or meaningfulness) of an example. We demonstrate the usefulness of this method in estimating the parameters of a latent variable classification model, and conduct experimental analyses on a range of synthetic and benchmark datasets. Furthermore, we theoretically analyze the performance of perceptron in this interactive learning framework.