Semi-Supervised Learning with Multiple Imputations on Non-Random Missing Labels

TL;DR

This paper introduces new semi-supervised learning methods that leverage multiple imputations and confidence thresholds to improve accuracy and reduce bias in datasets with non-random missing labels, especially under MNAR conditions.

Contribution

It proposes two novel methods combining multiple imputation models and confidence filtering, including SSL-DI, to address bias and improve performance in non-random missing label scenarios.

Findings

Outperforms existing methods in classification accuracy

Effectively reduces bias in MNAR and MCAR situations

Improves reliability of pseudo-labels through confidence thresholds

Abstract

Semi-Supervised Learning (SSL) is implemented when algorithms are trained on both labeled and unlabeled data. This is a very common application of ML as it is unrealistic to obtain a fully labeled dataset. Researchers have tackled three main issues: missing at random (MAR), missing completely at random (MCAR), and missing not at random (MNAR). The MNAR problem is the most challenging of the three as one cannot safely assume that all class distributions are equal. Existing methods, including Class-Aware Imputation (CAI) and Class-Aware Propensity (CAP), mostly overlook the non-randomness in the unlabeled data. This paper proposes two new methods of combining multiple imputation models to achieve higher accuracy and less bias. 1) We use multiple imputation models, create confidence intervals, and apply a threshold to ignore pseudo-labels with low confidence. 2) Our new method, SSL with De-biased Imputations (SSL-DI), aims to reduce bias by filtering out inaccurate data and finding a subset that is accurate and reliable. This subset of the larger dataset could be imputed into another SSL model, which will be less biased. The proposed models have been shown to be effective in both MCAR and MNAR situations, and experimental results show that our methodology outperforms existing methods in terms of classification accuracy and reducing bias.