Active Learning amidst Logical Knowledge

TL;DR

This paper investigates active learning for structured prediction with logical constraints, revealing limitations of uncertainty sampling and proposing new methods that outperform existing approaches across multiple datasets.

Contribution

It introduces novel active learning methods tailored for logical constraints in structured prediction, with theoretical insights and extensive empirical validation.

Findings

Uncertainty sampling is often ineffective for logical structured prediction.

Proposed methods outperform traditional active learning approaches.

Significant improvements demonstrated on ten diverse datasets.

Abstract

Structured prediction is ubiquitous in applications of machine learning such as knowledge extraction and natural language processing. Structure often can be formulated in terms of logical constraints. We consider the question of how to perform efficient active learning in the presence of logical constraints among variables inferred by different classifiers. We propose several methods and provide theoretical results that demonstrate the inappropriateness of employing uncertainty guided sampling, a commonly used active learning method. Furthermore, experiments on ten different datasets demonstrate that the methods significantly outperform alternatives in practice. The results are of practical significance in situations where labeled data is scarce.