Biologically-Motivated Deep Learning Method using Hierarchical Competitive Learning

TL;DR

This paper introduces a biologically-inspired unsupervised pre-training method for CNNs using hierarchical competitive learning, reducing reliance on labeled data and achieving competitive performance on image datasets.

Contribution

It presents a novel unsupervised pre-training approach for CNNs based on hierarchical competitive learning, enabling effective learning from unlabeled data.

Findings

Achieved state-of-the-art performance on ImageNet with the proposed method.

Enabled higher-level feature learning solely from forward signals without backpropagation.

Potential applicability to poorly labeled data like medical or time series data.

Abstract

This study proposes a novel biologically-motivated learning method for deep convolutional neural networks (CNNs). The combination of CNNs and back propagation (BP) learning is the most powerful method in recent machine learning regimes. However, it requires large labeled data for training, and this requirement can occasionally become a barrier for real world applications. To address this problem and utilize unlabeled data, I propose to introduce unsupervised competitive learning which only requires forward propagating signals as a pre-training method for CNNs. The method was evaluated by image discrimination tasks using MNIST, CIFAR-10, and ImageNet datasets, and it achieved a state-of-the-art performance as a biologically-motivated method in the ImageNet experiment. The results suggested that the method enables higher-level learning representations solely from forward propagating signals without a backward error signal for the learning of convolutional layers. The proposed method could be useful for a variety of poorly labeled data, for example, time series or medical data.