CaliMatch: Adaptive Calibration for Improving Safe Semi-supervised Learning

TL;DR

CaliMatch introduces an adaptive calibration method for safe semi-supervised learning, improving classifier and OOD detector calibration to reduce errors caused by overconfidence and enhance performance on multiple datasets.

Contribution

The paper proposes CaliMatch, a novel calibration approach with adaptive label smoothing and temperature scaling that eliminates manual tuning and improves safe SSL performance.

Findings

Outperforms existing safe SSL methods on multiple datasets

Effectively calibrates classifiers and OOD detectors to reduce errors

Theoretically justified importance of calibration in safe SSL

Abstract

Semi-supervised learning (SSL) uses unlabeled data to improve the performance of machine learning models when labeled data is scarce. However, its real-world applications often face the label distribution mismatch problem, in which the unlabeled dataset includes instances whose ground-truth labels are absent from the labeled training dataset. Recent studies, referred to as safe SSL, have addressed this issue by using both classification and out-of-distribution (OOD) detection. However, the existing methods may suffer from overconfidence in deep neural networks, leading to increased SSL errors because of high confidence in incorrect pseudo-labels or OOD detection. To address this, we propose a novel method, CaliMatch, which calibrates both the classifier and the OOD detector to foster safe SSL. CaliMatch presents adaptive label smoothing and temperature scaling, which eliminates the need to manually tune the smoothing degree for effective calibration. We give a theoretical justification for why improving the calibration of both the classifier and the OOD detector is crucial in safe SSL. Extensive evaluations on CIFAR-10, CIFAR-100, SVHN, TinyImageNet, and ImageNet demonstrate that CaliMatch outperforms the existing methods in safe SSL tasks.