Neuron-centric Hebbian Learning

TL;DR

This paper introduces Neuron-centric Hebbian Learning (NcHL), a simplified plasticity model focusing on neurons rather than synapses, reducing parameters significantly while maintaining performance in robotic tasks.

Contribution

The paper proposes NcHL, a novel neuron-focused plasticity model that simplifies optimization and reduces memory requirements compared to traditional synapse-focused rules.

Findings

NcHL performs comparably to ABCD rule in robotic tasks.

NcHL reduces parameters from 5W to 5N, significantly decreasing complexity.

The weightless NcHL model further decreases memory usage with minimal performance loss.

Abstract

One of the most striking capabilities behind the learning mechanisms of the brain is the adaptation, through structural and functional plasticity, of its synapses. While synapses have the fundamental role of transmitting information across the brain, several studies show that it is the neuron activations that produce changes on synapses. Yet, most plasticity models devised for artificial Neural Networks (NNs), e.g., the ABCD rule, focus on synapses, rather than neurons, therefore optimizing synaptic-specific Hebbian parameters. This approach, however, increases the complexity of the optimization process since each synapse is associated to multiple Hebbian parameters. To overcome this limitation, we propose a novel plasticity model, called Neuron-centric Hebbian Learning (NcHL), where optimization focuses on neuron- rather than synaptic-specific Hebbian parameters. Compared to the ABCD rule, NcHL reduces the parameters from $5W$ to $5N$, being $W$ and $N$ the number of weights and neurons, and usually $N \ll W$. We also devise a ``weightless'' NcHL model, which requires less memory by approximating the weights based on a record of neuron activations. Our experiments on two robotic locomotion tasks reveal that NcHL performs comparably to the ABCD rule, despite using up to $\sim97$ times less parameters, thus allowing for scalable plasticity