Statistical Mechanical Study on a Neural Network Model with Time Dependent Interactions

TL;DR

This paper investigates a neural network model with evolving neurons and synapses, revealing phase transition changes, increased stability of attractors, and effects of partial annealing through theoretical and numerical analysis.

Contribution

It introduces a model where synaptic interactions evolve with a Langevin equation, analyzing phase transitions and stability changes due to learning terms and partial annealing.

Findings

Phase transition changes from second to first order.

Increased stability of Hopfield attractor with higher learning coefficient.

Enhanced attractor stability after partial annealing.

Abstract

We study a neural network model in which both neurons and synaptic interactions evolve in time simultaneously. The time evolution of synaptic interactions is described by a Langevin equation including a Hebbian learning term, and a bias term which is the interactions of the Hopfield model. We assume that synaptic interactions change much slower than neurons and study the stationary states of synaptic interactions by the replica method. We find that the order of the phase transition changes from the second to the first and that the existence regions of the Hopfield attractor and mixed states increase as the coefficient of the learning term increases. We also study the AT stability of solutions and find that the temperature region in which the Hopfield attractor is stable increases as the learning coefficient increases. Theoretical results are confirmed by the direct numerical integration of the Langevin equation. Further, we study the characteristics of the resultant synaptic interactions by partial annealing and find that the stability of the attractor which emerges after partial annealing is enhanced and those of the coexistent attractors are reduced.