Constrained Labeling for Weakly Supervised Learning

TL;DR

This paper introduces a novel weak supervision method that uses constrained label spaces and random labeling to effectively combine noisy signals, improving classification performance with theoretical guarantees and empirical validation.

Contribution

It proposes a simple, data-free approach for weak supervision that leverages linear constraints to improve label aggregation and model accuracy.

Findings

Outperforms existing weak supervision methods on text and image tasks.

Converges quickly within a few gradient descent iterations.

Provides theoretical bounds on error reduction based on constraint rank.

Abstract

Curation of large fully supervised datasets has become one of the major roadblocks for machine learning. Weak supervision provides an alternative to supervised learning by training with cheap, noisy, and possibly correlated labeling functions from varying sources. The key challenge in weakly supervised learning is combining the different weak supervision signals while navigating misleading correlations in their errors. In this paper, we propose a simple data-free approach for combining weak supervision signals by defining a constrained space for the possible labels of the weak signals and training with a random labeling within this constrained space. Our method is efficient and stable, converging after a few iterations of gradient descent. We prove theoretical conditions under which the worst-case error of the randomized label decreases with the rank of the linear constraints. We show experimentally that our method outperforms other weak supervision methods on various text- and image-classification tasks.