Pattern reconstruction and sequence processing in feed-forward layered neural networks near saturation

TL;DR

This paper analyzes the dynamics of pattern reconstruction and sequence processing in layered neural networks near saturation, extending previous models to include stochastic noise and exploring phase diagrams and solution behaviors.

Contribution

It introduces an exactly solvable model that incorporates finite stochastic noise in layered neural networks, extending prior work to near-saturation conditions.

Findings

Phase diagrams with stationary and quasi-periodic solutions

Dependence of solutions on stochastic noise and initial inputs

Extension of earlier models to include finite noise effects

Abstract

The dynamics and the stationary states for the competition between pattern reconstruction and asymmetric sequence processing are studied here in an exactly solvable feed-forward layered neural network model of binary units and patterns near saturation. Earlier work by Coolen and Sherrington on a parallel dynamics far from saturation is extended here to account for finite stochastic noise due to a Hebbian and a sequential learning rule. Phase diagrams are obtained with stationary states and quasi-periodic non-stationary solutions. The relevant dependence of these diagrams and of the quasi-periodic solutions on the stochastic noise and on initial inputs for the overlaps is explicitly discussed.