Prediction-Powered Semi-Supervised Learning with Online Power Tuning

TL;DR

This paper introduces a novel semi-supervised learning method that uses an unbiased gradient estimator and online tuning of pseudo-label contributions, leading to improved model performance.

Contribution

It extends Prediction-Powered Inference to semi-supervised learning with an unbiased gradient estimator and online tuning of pseudo-label influence.

Findings

Outperforms classic SSL baselines in experiments

Demonstrates effectiveness of online tuning of pseudo-label contribution

Validates approach on synthetic and real datasets

Abstract

Prediction-Powered Inference (PPI) is a recently proposed statistical inference technique for parameter estimation that leverages pseudo-labels on both labeled and unlabeled data to construct an unbiased, low-variance estimator. In this work, we extend its core idea to semi-supervised learning (SSL) for model training, introducing a novel unbiased gradient estimator. This extension addresses a key challenge in SSL: while unlabeled data can improve model performance, its benefit heavily depends on the quality of pseudo-labels. Inaccurate pseudo-labels can introduce bias, leading to suboptimal models.To balance the contributions of labeled and pseudo-labeled data, we utilize an interpolation parameter and tune it on the fly, alongside the model parameters, using a one-dimensional online learning algorithm. We verify the practical advantage of our approach through experiments on both synthetic and real datasets, demonstrating improved performance over classic SSL baselines and PPI methods that tune the interpolation parameter offline.