Reduction-based Pseudo-label Generation for Instance-dependent Partial Label Learning

TL;DR

This paper introduces a reduction-based pseudo-label generation method for instance-dependent partial label learning, aiming to improve model training by reducing overfitting to incorrect candidate labels.

Contribution

It proposes a novel reduction-based pseudo-label generation technique using multi-branch auxiliary models to mitigate overfitting in ID-PLL.

Findings

Pseudo-labels are more consistent with the Bayes optimal classifier.

The method effectively reduces the impact of incorrect candidate labels.

Experimental results demonstrate improved learning performance.

Abstract

Instance-dependent Partial Label Learning (ID-PLL) aims to learn a multi-class predictive model given training instances annotated with candidate labels related to features, among which correct labels are hidden fixed but unknown. The previous works involve leveraging the identification capability of the training model itself to iteratively refine supervision information. However, these methods overlook a critical aspect of ID-PLL: the training model is prone to overfitting on incorrect candidate labels, thereby providing poor supervision information and creating a bottleneck in training. In this paper, we propose to leverage reduction-based pseudo-labels to alleviate the influence of incorrect candidate labels and train our predictive model to overcome this bottleneck. Specifically, reduction-based pseudo-labels are generated by performing weighted aggregation on the outputs of a multi-branch auxiliary model, with each branch trained in a label subspace that excludes certain labels. This approach ensures that each branch explicitly avoids the disturbance of the excluded labels, allowing the pseudo-labels provided for instances troubled by these excluded labels to benefit from the unaffected branches. Theoretically, we demonstrate that reduction-based pseudo-labels exhibit greater consistency with the Bayes optimal classifier compared to pseudo-labels directly generated from the predictive model.