A mathematical analysis of the effects of Hebbian learning rules on the dynamics and structure of discrete-time random recurrent neural networks

TL;DR

This paper provides a mathematical framework to understand how Hebbian learning influences the dynamics and structure of discrete-time random recurrent neural networks, revealing conditions for stability and sensitivity to learned patterns.

Contribution

It offers a novel mathematical analysis of Hebbian learning effects, incorporating passive forgetting and multiple time scales, linking neuronal dynamics with network structure through Jacobian matrices.

Findings

Hebbian learning can transition networks from chaos to steady states via bifurcations.

Sensitivity to learned patterns peaks when the largest Lyapunov exponent approaches zero.

The analysis connects network stability with the spectral properties of Jacobian matrices.

Abstract

We present a mathematical analysis of the effects of Hebbian learning in random recurrent neural networks, with a generic Hebbian learning rule including passive forgetting and different time scales for neuronal activity and learning dynamics. Previous numerical works have reported that Hebbian learning drives the system from chaos to a steady state through a sequence of bifurcations. Here, we interpret these results mathematically and show that these effects, involving a complex coupling between neuronal dynamics and synaptic graph structure, can be analyzed using Jacobian matrices, which introduce both a structural and a dynamical point of view on the neural network evolution. Furthermore, we show that the sensitivity to a learned pattern is maximal when the largest Lyapunov exponent is close to 0. We discuss how neural networks may take advantage of this regime of high functional interest.