Biologically-inspired neuronal adaptation improves learning in neural networks

TL;DR

This paper introduces a biologically-inspired neuronal adaptation mechanism to enhance the learning capabilities of neural networks, demonstrating improved performance on standard datasets by augmenting existing learning algorithms.

Contribution

It proposes a novel adaptation feature inspired by neurons, integrated into CHL and EP algorithms, leading to performance improvements in neural network training.

Findings

Adaptation improved network accuracy on MNIST and CIFAR-10.

Biological inspiration enhances stability and learning performance.

Neuronal adaptation could be a key brain mechanism for learning.

Abstract

Since humans still outperform artificial neural networks on many tasks, drawing inspiration from the brain may help to improve current machine learning algorithms. Contrastive Hebbian Learning (CHL) and Equilibrium Propagation (EP) are biologically plausible algorithms that update weights using only local information (without explicitly calculating gradients) and still achieve performance comparable to conventional backpropagation. In this study, we augmented CHL and EP with Adjusted Adaptation, inspired by the adaptation effect observed in neurons, in which a neuron's response to a given stimulus is adjusted after a short time. We add this adaptation feature to multilayer perceptrons and convolutional neural networks trained on MNIST and CIFAR-10. Surprisingly, adaptation improved the performance of these networks. We discuss the biological inspiration for this idea and investigate why Neuronal Adaptation could be an important brain mechanism to improve the stability and accuracy of learning.