Online Representation Learning with Single and Multi-layer Hebbian Networks for Image Classification

TL;DR

This paper evaluates a Hebbian-like unsupervised learning rule for image feature extraction, demonstrating its effectiveness in classification tasks on CIFAR-10 and its potential for online learning networks.

Contribution

It introduces a novel Hebbian-based learning rule derived from a multidimensional scaling cost-function for image classification.

Findings

Performs well compared to other unsupervised algorithms

Effective in single and multi-layer architectures

Suitable for online learning applications

Abstract

Unsupervised learning permits the development of algorithms that are able to adapt to a variety of different data sets using the same underlying rules thanks to the autonomous discovery of discriminating features during training. Recently, a new class of Hebbian-like and local unsupervised learning rules for neural networks have been developed that minimise a similarity matching cost-function. These have been shown to perform sparse representation learning. This study tests the effectiveness of one such learning rule for learning features from images. The rule implemented is derived from a nonnegative classical multidimensional scaling cost-function, and is applied to both single and multi-layer architectures. The features learned by the algorithm are then used as input to an SVM to test their effectiveness in classification on the established CIFAR-10 image dataset. The algorithm performs well in comparison to other unsupervised learning algorithms and multi-layer networks, thus suggesting its validity in the design of a new class of compact, online learning networks.