Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks

TL;DR

This paper develops a biologically plausible Hebbian learning-based CNN architecture that matches backpropagation accuracy on CIFAR-10 and improves the realism of neural network models.

Contribution

It introduces an optimal Hebbian CNN architecture integrating competition mechanisms, achieving competitive accuracy and biological plausibility.

Findings

Achieved 75.2% accuracy on CIFAR-10, matching backpropagation.

Surpassed state-of-the-art hard-WTA CNNs by 10.6%.

Demonstrated sparse hierarchical learning with complex receptive fields.

Abstract

The research presented in this paper advances the integration of Hebbian learning into Convolutional Neural Networks (CNNs) for image processing, systematically exploring different architectures to build an optimal configuration, adhering to biological tenability. Hebbian learning operates on local unsupervised neural information to form feature representations, providing an alternative to the popular but arguably biologically implausible and computationally intensive backpropagation learning algorithm. The suggested optimal architecture significantly enhances recent research aimed at integrating Hebbian learning with competition mechanisms and CNNs, expanding their representational capabilities by incorporating hard Winner-Takes-All (WTA) competition, Gaussian lateral inhibition mechanisms, and Bienenstock-Cooper-Munro (BCM) learning rule in a single model. Mean accuracy classification measures during the last half of test epochs on CIFAR-10 revealed that the resulting optimal model matched its end-to-end backpropagation variant with 75.2% each, critically surpassing the state-of-the-art hard-WTA performance in CNNs of the same network depth (64.6%) by 10.6%. It also achieved competitive performance on MNIST (98%) and STL-10 (69.5%). Moreover, results showed clear indications of sparse hierarchical learning through increasingly complex and abstract receptive fields. In summary, our implementation enhances both the performance and the generalisability of the learnt representations and constitutes a crucial step towards more biologically realistic artificial neural networks.