Inferring Generative Model Structure with Static Analysis

TL;DR

Coral is a static analysis-based method that infers the structure of generative models from weak supervision code, significantly reducing data needs and improving label quality in machine learning tasks.

Contribution

Coral introduces a novel static analysis approach to infer generative model structure, reducing sample complexity and enhancing performance over traditional methods.

Findings

Coral matches or exceeds traditional structure learning by up to 3.81 F1 points.

Using Coral's inferred structure improves radiology data labeling accuracy by 3.07 points.

Coral's sample complexity scales quasilinearly with heuristics, outperforming exponential scaling.

Abstract

Obtaining enough labeled data to robustly train complex discriminative models is a major bottleneck in the machine learning pipeline. A popular solution is combining multiple sources of weak supervision using generative models. The structure of these models affects training label quality, but is difficult to learn without any ground truth labels. We instead rely on these weak supervision sources having some structure by virtue of being encoded programmatically. We present Coral, a paradigm that infers generative model structure by statically analyzing the code for these heuristics, thus reducing the data required to learn structure significantly. We prove that Coral's sample complexity scales quasilinearly with the number of heuristics and number of relations found, improving over the standard sample complexity, which is exponential in $n$ for identifying $n^{\textrm{th}}$ degree relations. Experimentally, Coral matches or outperforms traditional structure learning approaches by up to 3.81 F1 points. Using Coral to model dependencies instead of assuming independence results in better performance than a fully supervised model by 3.07 accuracy points when heuristics are used to label radiology data without ground truth labels.