Feed-Forward Chains of Recurrent Attractor Neural Networks Near Saturation

TL;DR

This paper analyzes a layered neural network model combining recurrent and feed-forward Hebbian interactions, revealing phase transitions caused by the competition between ergodic and non-ergodic modes of operation.

Contribution

It introduces a new interpolating model between fully recurrent and feed-forward attractor networks, analyzed via replica methods at zero temperature.

Findings

Identification of phase transitions due to mode competition

Analysis of the interplay between recurrent and feed-forward dynamics

Insights into the network's ergodic and non-ergodic behavior

Abstract

We perform a stationary state replica analysis for a layered network of Ising spin neurons, with recurrent Hebbian interactions within each layer, in combination with strictly feed-forward Hebbian interactions between successive layers. This model interpolates between the fully recurrent and symmetric attractor network studied by Amit el al, and the strictly feed-forward attractor network studied by Domany et al. Due to the absence of detailed balance, it is as yet solvable only in the zero temperature limit. The built-in competition between two qualitatively different modes of operation, feed-forward (ergodic within layers) versus recurrent (non- ergodic within layers), is found to induce interesting phase transitions.