Forming Auxiliary High-confident Instance-level Loss to Promote Learning from Label Proportions

TL;DR

This paper introduces L^2P-AHIL, a novel method for learning from label proportions that uses a dual entropy-based weight to improve pseudo-label confidence, especially for large bags, enhancing classifier performance.

Contribution

It proposes a dual entropy-based weight to adaptively assess pseudo-label confidence, forming a high-confident instance-level loss to improve LLP training.

Findings

Outperforms existing methods on benchmark datasets.

Performance improves with larger bag sizes.

Effectively reduces pseudo-label inaccuracies.

Abstract

Learning from label proportions (LLP), i.e., a challenging weakly-supervised learning task, aims to train a classifier by using bags of instances and the proportions of classes within bags, rather than annotated labels for each instance. Beyond the traditional bag-level loss, the mainstream methodology of LLP is to incorporate an auxiliary instance-level loss with pseudo-labels formed by predictions. Unfortunately, we empirically observed that the pseudo-labels are are often inaccurate due to over-smoothing, especially for the scenarios with large bag sizes, hurting the classifier induction. To alleviate this problem, we suggest a novel LLP method, namely Learning from Label Proportions with Auxiliary High-confident Instance-level Loss (L^2P-AHIL). Specifically, we propose a dual entropy-based weight (DEW) method to adaptively measure the confidences of pseudo-labels. It simultaneously emphasizes accurate predictions at the bag level and avoids overly smoothed predictions. We then form high-confident instance-level loss with DEW, and jointly optimize it with the bag-level loss in a self-training manner. The experimental results on benchmark datasets show that L^2P-AHIL can surpass the existing baseline methods, and the performance gain can be more significant as the bag size increases. The implementation of our method is available at https://github.com/TianhaoMa5/LLP-AHIL.