Asynchronous Hebbian/anti-Hebbian networks

TL;DR

This paper introduces a biologically plausible Hebbian learning rule enabling real-time, continuous learning of factorized representations in neural networks, while preventing catastrophic forgetting across different stimuli distributions.

Contribution

It proposes a new Hebbian learning rule that allows for efficient, continuous learning and biological plausibility, addressing limitations of previous models requiring stable state convergence.

Findings

Enables time-continuous learning of factorized representations

Prevents catastrophic forgetting in sequential stimulus learning

Achieves similar results to classic Hebbian/anti-Hebbian learning

Abstract

Lateral inhibition models coupled with Hebbian plasticity have been shown to learn factorised causal representations of input stimuli, for instance, oriented edges are learned from natural images. Currently, these models require the recurrent dynamics to settle into a stable state before weight changes can be applied, which is not only biologically implausible, but also impractical for real-time learning systems. Here, we propose a new Hebbian learning rule which is implemented using plausible biological mechanisms that have been observed experimentally. We find that this rule allows for efficient, time-continuous learning of factorised representations, very similar to the classic noncontinuous Hebbian/anti-Hebbian learning. Furthermore, we show that this rule naturally prevents catastrophic forgetting when stimuli from different distributions are shown sequentially.