Easy Learning from Label Proportions

TL;DR

This paper introduces Easyllp, a simple and flexible debiasing method for Learning from Label Proportions that accurately estimates individual instance loss, improving performance with theoretical guarantees and empirical validation.

Contribution

We propose Easyllp, a novel aggregate-label-based debiasing approach applicable to various learning frameworks, with provable guarantees and variance reduction techniques.

Findings

Achieves comparable or better performance than previous LLP methods.

Provides theoretical guarantees on instance-level performance.

Demonstrates effectiveness across multiple datasets.

Abstract

We consider the problem of Learning from Label Proportions (LLP), a weakly supervised classification setup where instances are grouped into "bags", and only the frequency of class labels at each bag is available. Albeit, the objective of the learner is to achieve low task loss at an individual instance level. Here we propose Easyllp: a flexible and simple-to-implement debiasing approach based on aggregate labels, which operates on arbitrary loss functions. Our technique allows us to accurately estimate the expected loss of an arbitrary model at an individual level. We showcase the flexibility of our approach by applying it to popular learning frameworks, like Empirical Risk Minimization (ERM) and Stochastic Gradient Descent (SGD) with provable guarantees on instance level performance. More concretely, we exhibit a variance reduction technique that makes the quality of LLP learning deteriorate only by a factor of k (k being bag size) in both ERM and SGD setups, as compared to full supervision. Finally, we validate our theoretical results on multiple datasets demonstrating our algorithm performs as well or better than previous LLP approaches in spite of its simplicity.