Billion-scale semi-supervised learning for image classification

TL;DR

This paper demonstrates that semi-supervised learning with a billion unlabelled images significantly improves image classification accuracy, achieving state-of-the-art results on ImageNet using standard architectures.

Contribution

It introduces a large-scale semi-supervised learning pipeline leveraging a billion unlabelled images, with analysis and recommendations for high-accuracy models.

Findings

ResNet-50 achieves 81.2% top-1 accuracy on ImageNet.

Large unlabelled datasets substantially boost model performance.

The approach benefits image, video, and fine-grained classification tasks.

Abstract

This paper presents a study of semi-supervised learning with large convolutional networks. We propose a pipeline, based on a teacher/student paradigm, that leverages a large collection of unlabelled images (up to 1 billion). Our main goal is to improve the performance for a given target architecture, like ResNet-50 or ResNext. We provide an extensive analysis of the success factors of our approach, which leads us to formulate some recommendations to produce high-accuracy models for image classification with semi-supervised learning. As a result, our approach brings important gains to standard architectures for image, video and fine-grained classification. For instance, by leveraging one billion unlabelled images, our learned vanilla ResNet-50 achieves 81.2% top-1 accuracy on the ImageNet benchmark.