Meta Pseudo Labels

TL;DR

Meta Pseudo Labels is a semi-supervised learning approach that uses an adaptive teacher network, improving pseudo label quality and achieving state-of-the-art accuracy on ImageNet.

Contribution

It introduces a dynamic teacher network that adapts based on student performance, enhancing pseudo label quality over fixed-teacher methods.

Findings

Achieved 90.2% top-1 accuracy on ImageNet.

Outperformed previous semi-supervised methods by 1.6%.

Demonstrated effectiveness of adaptive teacher in semi-supervised learning.

Abstract

We present Meta Pseudo Labels, a semi-supervised learning method that achieves a new state-of-the-art top-1 accuracy of 90.2% on ImageNet, which is 1.6% better than the existing state-of-the-art. Like Pseudo Labels, Meta Pseudo Labels has a teacher network to generate pseudo labels on unlabeled data to teach a student network. However, unlike Pseudo Labels where the teacher is fixed, the teacher in Meta Pseudo Labels is constantly adapted by the feedback of the student's performance on the labeled dataset. As a result, the teacher generates better pseudo labels to teach the student. Our code will be available at https://github.com/google-research/google-research/tree/master/meta_pseudo_labels.