A Unified and Stable Risk Minimization Framework for Weakly Supervised Learning with Theoretical Guarantees

TL;DR

This paper introduces a unified, stable risk minimization framework for various weakly supervised learning settings, providing theoretical guarantees and demonstrating consistent empirical improvements without heuristic stabilization.

Contribution

It proposes a single optimization-based framework that subsumes multiple weak supervision types and offers theoretical analysis including generalization bounds and robustness to class-prior misspecification.

Findings

Consistent empirical gains across datasets and supervision types.

Robustness to class-prior misspecification and overfitting.

Theoretical guarantees including generalization bounds and identifiability conditions.

Abstract

Weakly supervised learning has emerged as a practical alternative to fully supervised learning when complete and accurate labels are costly or infeasible to acquire. However, many existing methods are tailored to specific supervision patterns -- such as positive-unlabeled (PU), unlabeled-unlabeled (UU), complementary-label (CLL), partial-label (PLL), or similarity-unlabeled annotations -- and rely on post-hoc corrections to mitigate instability induced by indirect supervision. We propose a principled, unified framework that bypasses such post-hoc adjustments by directly formulating a stable surrogate risk grounded in the structure of weakly supervised data. The formulation naturally subsumes diverse settings -- including PU, UU, CLL, PLL, multi-class unlabeled, and tuple-based learning -- under a single optimization objective. We further establish a non-asymptotic generalization bound via Rademacher complexity that clarifies how supervision structure, model capacity, and sample size jointly govern performance. Beyond this, we analyze the effect of class-prior misspecification on the bound, deriving explicit terms that quantify its impact, and we study identifiability, giving sufficient conditions -- most notably via supervision stratification across groups -- under which the target risk is recoverable. Extensive experiments show consistent gains across class priors, dataset scales, and class counts -- without heuristic stabilization -- while exhibiting robustness to overfitting.