A Review of Pseudo-Labeling for Computer Vision

TL;DR

This paper reviews pseudo-labeling techniques in computer vision, exploring their applications across semi-supervised, self-supervised, and unsupervised learning, and discusses how cross-area insights can lead to new research directions.

Contribution

It broadens the understanding of pseudo-labeling beyond semi-supervised learning, connecting it with self-supervised and unsupervised methods to identify new research opportunities.

Findings

Pseudo-labeling is applicable across multiple learning paradigms.

Connections between different areas can enhance pseudo-labeling techniques.

Identifies promising future research directions in curriculum learning and regularization.

Abstract

Deep neural models have achieved state of the art performance on a wide range of problems in computer science, especially in computer vision. However, deep neural networks often require large datasets of labeled samples to generalize effectively, and an important area of active research is semi-supervised learning, which attempts to instead utilize large quantities of (easily acquired) unlabeled samples. One family of methods in this space is pseudo-labeling, a class of algorithms that use model outputs to assign labels to unlabeled samples which are then used as labeled samples during training. Such assigned labels, called pseudo-labels, are most commonly associated with the field of semi-supervised learning. In this work we explore a broader interpretation of pseudo-labels within both self-supervised and unsupervised methods. By drawing the connection between these areas we identify new directions when advancements in one area would likely benefit others, such as curriculum learning and self-supervised regularization.