Unsupervised Visual Representation Learning with Increasing Object Shape Bias

TL;DR

This paper proposes a novel unsupervised learning method for visual representations using contrastive predictive coding, enabling training on unlimited non-annotated images and achieving state-of-the-art performance.

Contribution

It introduces a new unsupervised learning approach for computer vision that leverages contrastive predictive coding to improve performance without labeled data.

Findings

Achieves state-of-the-art results with unsupervised learning

Enables training on unlimited non-annotated images

Paves the way for universal large-scale pre-trained vision models

Abstract

(Very early draft)Traditional supervised learning keeps pushing convolution neural network(CNN) achieving state-of-art performance. However, lack of large-scale annotation data is always a big problem due to the high cost of it, even ImageNet dataset is over-fitted by complex models now. The success of unsupervised learning method represented by the Bert model in natural language processing(NLP) field shows its great potential. And it makes that unlimited training samples becomes possible and the great universal generalization ability changes NLP research direction directly. In this article, we purpose a novel unsupervised learning method based on contrastive predictive coding. Under that, we are able to train model with any non-annotation images and improve model's performance to reach state-of-art performance at the same level of model complexity. Beside that, since the number of training images could be unlimited amplification, an universal large-scale pre-trained computer vision model is possible in the future.