CalibrateMix: Guided-Mixup Calibration of Image Semi-Supervised Models

TL;DR

CalibrateMix is a novel semi-supervised learning technique that improves model calibration and accuracy by using targeted mixup based on training dynamics to better handle easy and hard samples.

Contribution

This work introduces CalibrateMix, a new mixup-based method that enhances calibration and accuracy in semi-supervised image classification by leveraging training dynamics.

Findings

Achieves lower expected calibration error (ECE) compared to existing SSL methods.

Maintains or improves classification accuracy.

Effective across multiple benchmark datasets.

Abstract

Semi-supervised learning (SSL) has demonstrated high performance in image classification tasks by effectively utilizing both labeled and unlabeled data. However, existing SSL methods often suffer from poor calibration, with models yielding overconfident predictions that misrepresent actual prediction likelihoods. Recently, neural networks trained with {\tt mixup} that linearly interpolates random examples from the training set have shown better calibration in supervised settings. However, calibration of neural models remains under-explored in semi-supervised settings. Although effective in supervised model calibration, random mixup of pseudolabels in SSL presents challenges due to the overconfidence and unreliability of pseudolabels. In this work, we introduce CalibrateMix, a targeted mixup-based approach that aims to improve the calibration of SSL models while maintaining or even improving their classification accuracy. Our method leverages training dynamics of labeled and unlabeled samples to identify ``easy-to-learn'' and ``hard-to-learn'' samples, which in turn are utilized in a targeted mixup of easy and hard samples. Experimental results across several benchmark image datasets show that our method achieves lower expected calibration error (ECE) and superior accuracy compared to existing SSL approaches.