Local Contrast Learning

TL;DR

This paper introduces Local Contrast Learning (LCL), a novel deep learning approach inspired by human cognition, that effectively enables one-shot classification with small datasets, outperforming existing methods and even human accuracy.

Contribution

LCL is a new deep learning technique that uses contrastive samples to improve one-shot learning, reducing overfitting and achieving high accuracy on small datasets.

Findings

Achieved 97.99% accuracy on Omniglot one-shot classification

Outperformed human performance and Bayesian Program Learning

Demonstrated effectiveness with a 122-layer model on limited data

Abstract

Learning a deep model from small data is yet an opening and challenging problem. We focus on one-shot classification by deep learning approach based on a small quantity of training samples. We proposed a novel deep learning approach named Local Contrast Learning (LCL) based on the key insight about a human cognitive behavior that human recognizes the objects in a specific context by contrasting the objects in the context or in her/his memory. LCL is used to train a deep model that can contrast the recognizing sample with a couple of contrastive samples randomly drawn and shuffled. On one-shot classification task on Omniglot, the deep model based LCL with 122 layers and 1.94 millions of parameters, which was trained on a tiny dataset with only 60 classes and 20 samples per class, achieved the accuracy 97.99% that outperforms human and state-of-the-art established by Bayesian Program Learning (BPL) trained on 964 classes. LCL is a fundamental idea which can be applied to alleviate parametric model's overfitting resulted by lack of training samples.