Not All Unlabeled Data are Equal: Learning to Weight Data in Semi-supervised Learning

TL;DR

This paper introduces a novel semi-supervised learning approach that assigns individual weights to unlabeled data points using influence functions, improving performance over existing methods.

Contribution

It proposes a new method to dynamically weight unlabeled examples in SSL using influence functions, replacing manual tuning of weights.

Findings

Outperforms state-of-the-art SSL methods on image classification tasks.

Efficient approximation of influence functions enables scalable training.

Improves model performance by selectively weighting unlabeled data.

Abstract

Existing semi-supervised learning (SSL) algorithms use a single weight to balance the loss of labeled and unlabeled examples, i.e., all unlabeled examples are equally weighted. But not all unlabeled data are equal. In this paper we study how to use a different weight for every unlabeled example. Manual tuning of all those weights -- as done in prior work -- is no longer possible. Instead, we adjust those weights via an algorithm based on the influence function, a measure of a model's dependency on one training example. To make the approach efficient, we propose a fast and effective approximation of the influence function. We demonstrate that this technique outperforms state-of-the-art methods on semi-supervised image and language classification tasks.