Learning from Label Proportions with Instance-wise Consistency

TL;DR

This paper introduces statistically grounded methods for learning from label proportions that ensure consistency and reduce computational complexity, improving instance classification accuracy in weakly supervised settings.

Contribution

It proposes new risk- and classifier-consistent learning methods for LLP with theoretical guarantees and a heuristic to lower computational costs.

Findings

Methods are risk- and classifier-consistent in an instance-wise manner.

The proposed methods outperform existing approaches on benchmark datasets.

The heuristic approximation effectively reduces computational complexity.

Abstract

Learning from Label Proportions (LLP) is a weakly supervised learning method that aims to perform instance classification from training data consisting of pairs of bags containing multiple instances and the class label proportions within the bags. Previous studies on multiclass LLP can be divided into two categories according to the learning task: per-instance label classification and per-bag label proportion estimation. However, these methods often results in high variance estimates of the risk when applied to complex models, or lack statistical learning theory arguments. To address this issue, we propose new learning methods based on statistical learning theory for both per-instance and per-bag policies. We demonstrate that the proposed methods are respectively risk-consistent and classifier-consistent in an instance-wise manner, and analyze the estimation error bounds. Additionally, we present a heuristic approximation method that utilizes an existing method for regressing label proportions to reduce the computational complexity of the proposed methods. Through benchmark experiments, we demonstrated the effectiveness of the proposed methods.