Semi-Supervised Learning with Declaratively Specified Entropy Constraints

TL;DR

This paper introduces a declarative framework for semi-supervised learning that allows flexible specification and combination of multiple SSL heuristics, leading to improved performance on benchmark tasks.

Contribution

It presents a novel declarative approach to specify and combine SSL strategies, including heuristics and agreement constraints, with automatic optimization for enhanced results.

Findings

Achieved state-of-the-art on relation extraction benchmark

Demonstrated consistent improvements across SSL benchmarks

Enabled flexible modeling of SSL heuristics and their combinations

Abstract

We propose a technique for declaratively specifying strategies for semi-supervised learning (SSL). The proposed method can be used to specify ensembles of semi-supervised learning, as well as agreement constraints and entropic regularization constraints between these learners, and can be used to model both well-known heuristics such as co-training and novel domain-specific heuristics. In addition to representing individual SSL heuristics, we show that multiple heuristics can also be automatically combined using Bayesian optimization methods. We show consistent improvements on a suite of well-studied SSL benchmarks, including a new state-of-the-art result on a difficult relation extraction task.