Efficient and Reliable Probabilistic Interactive Learning with Structured Outputs

TL;DR

This paper introduces CRISPs, a class of probabilistic models that enable efficient and reliable active and skeptical learning in large structured output spaces by ensuring tractable probabilistic computations.

Contribution

The paper defines CRISPs, a new class of probabilistic models that balance expressiveness with computational tractability for interactive learning scenarios.

Findings

CRISPs enable scalable active learning in structured output spaces.

CRISPs support skeptical learning with noisy labels effectively.

The approach maintains computational efficiency while preserving model expressiveness.

Abstract

In this position paper, we study interactive learning for structured output spaces, with a focus on active learning, in which labels are unknown and must be acquired, and on skeptical learning, in which the labels are noisy and may need relabeling. These scenarios require expressive models that guarantee reliable and efficient computation of probabilistic quantities to measure uncertainty. We identify conditions under which a class of probabilistic models -- which we denote CRISPs -- meet all of these conditions, thus delivering tractable computation of the above quantities while preserving expressiveness. Building on prior work on tractable probabilistic circuits, we illustrate how CRISPs enable robust and efficient active and skeptical learning in large structured output spaces.