Resolving label uncertainty with implicit posterior models

TL;DR

This paper introduces a unified method for inferring labels in datasets with uncertain or weak prior information by leveraging an implicit generative model framework, applicable across various machine learning tasks.

Contribution

It develops a novel training objective that enables learning from weak beliefs, noisy labels, and auxiliary information within a unified implicit posterior modeling approach.

Findings

Effective in classification with negative examples

Successful in learning from rankings and weak supervision

Applicable to diverse domains like imagery and text

Abstract

We propose a method for jointly inferring labels across a collection of data samples, where each sample consists of an observation and a prior belief about the label. By implicitly assuming the existence of a generative model for which a differentiable predictor is the posterior, we derive a training objective that allows learning under weak beliefs. This formulation unifies various machine learning settings; the weak beliefs can come in the form of noisy or incomplete labels, likelihoods given by a different prediction mechanism on auxiliary input, or common-sense priors reflecting knowledge about the structure of the problem at hand. We demonstrate the proposed algorithms on diverse problems: classification with negative training examples, learning from rankings, weakly and self-supervised aerial imagery segmentation, co-segmentation of video frames, and coarsely supervised text classification.