Two-stage Training for Learning from Label Proportions

TL;DR

This paper proposes a two-stage training framework for learning from label proportions that improves instance-level classification accuracy by enforcing strict proportion consistency and reducing label noise.

Contribution

It introduces a constrained optimization approach with mixup and symmetric crossentropy to enhance existing deep LLP models, addressing issues of high-entropy distributions and proportion mismatch.

Findings

Significant performance improvements on benchmark datasets

Effective reduction of label noise and distribution mismatch

Framework is model-agnostic and applicable as a post-hoc phase

Abstract

Learning from label proportions (LLP) aims at learning an instance-level classifier with label proportions in grouped training data. Existing deep learning based LLP methods utilize end-to-end pipelines to obtain the proportional loss with Kullback-Leibler divergence between the bag-level prior and posterior class distributions. However, the unconstrained optimization on this objective can hardly reach a solution in accordance with the given proportions. Besides, concerning the probabilistic classifier, this strategy unavoidably results in high-entropy conditional class distributions at the instance level. These issues further degrade the performance of the instance-level classification. In this paper, we regard these problems as noisy pseudo labeling, and instead impose the strict proportion consistency on the classifier with a constrained optimization as a continuous training stage for existing LLP classifiers. In addition, we introduce the mixup strategy and symmetric crossentropy to further reduce the label noise. Our framework is model-agnostic, and demonstrates compelling performance improvement in extensive experiments, when incorporated into other deep LLP models as a post-hoc phase.