Emerging NeoHebbian Dynamics in Forward-Forward Learning: Implications for Neuromorphic Computing

TL;DR

This paper demonstrates that the Forward-Forward Algorithm (FFA) is equivalent to a neo-Hebbian learning rule when using a squared Euclidean norm as a goodness function, linking biological learning principles with practical neural network training.

Contribution

It establishes the equivalence between FFA and neo-Hebbian learning, and compares their performance in analog and spiking neural networks, highlighting implications for neuromorphic computing.

Findings

FFA produces similar accuracy in analog and spiking neural networks.

Empirical evidence links biological Hebbian rules with FFA training.

Analog networks trained with FFA are suitable for neuromorphic applications.

Abstract

Advances in neural computation have predominantly relied on the gradient backpropagation algorithm (BP). However, the recent shift towards non-stationary data modeling has highlighted the limitations of this heuristic, exposing that its adaptation capabilities are far from those seen in biological brains. Unlike BP, where weight updates are computed through a reverse error propagation path, Hebbian learning dynamics provide synaptic updates using only information within the layer itself. This has spurred interest in biologically plausible learning algorithms, hypothesized to overcome BP's shortcomings. In this context, Hinton recently introduced the Forward-Forward Algorithm (FFA), which employs local learning rules for each layer and has empirically proven its efficacy in multiple data modeling tasks. In this work we argue that when employing a squared Euclidean norm as a goodness function driving the local learning, the resulting FFA is equivalent to a neo-Hebbian Learning Rule. To verify this result, we compare the training behavior of FFA in analog networks with its Hebbian adaptation in spiking neural networks. Our experiments demonstrate that both versions of FFA produce similar accuracy and latent distributions. The findings herein reported provide empirical evidence linking biological learning rules with currently used training algorithms, thus paving the way towards extrapolating the positive outcomes from FFA to Hebbian learning rules. Simultaneously, our results imply that analog networks trained under FFA could be directly applied to neuromorphic computing, leading to reduced energy usage and increased computational speed.