Spatial contrasting for deep unsupervised learning

TL;DR

This paper introduces a novel unsupervised training method for convolutional networks that leverages spatial region contrasting within images, compatible with standard training techniques and enhancing visual task performance.

Contribution

The paper proposes a new spatial contrasting approach for unsupervised learning of convolutional networks, compatible with existing architectures and training methods.

Findings

Enables unsupervised training of convolutional networks using spatial region contrasting.

Can be integrated with standard neural network training techniques like SGD and back-propagation.

Provides a complementary approach to supervised learning for visual tasks.

Abstract

Convolutional networks have marked their place over the last few years as the best performing model for various visual tasks. They are, however, most suited for supervised learning from large amounts of labeled data. Previous attempts have been made to use unlabeled data to improve model performance by applying unsupervised techniques. These attempts require different architectures and training methods. In this work we present a novel approach for unsupervised training of Convolutional networks that is based on contrasting between spatial regions within images. This criterion can be employed within conventional neural networks and trained using standard techniques such as SGD and back-propagation, thus complementing supervised methods.