Learning by a nerual net in a noisy environment - The pseudo-inverse solution revisited

TL;DR

This paper analyzes a recurrent neural network that learns patterns in noisy environments using a Hebbian learning rule, revisiting the pseudo-inverse solution and exploring the effects of noise on weight stability.

Contribution

It provides an explicit expression for the expected weights in noisy conditions and discusses the stability properties of the neural system.

Findings

Expected weights differ from the pseudo-inverse solution under noise.

Explicit formula for weights' expectation in noisy learning.

Stability analysis of the neural network system.

Abstract

A recurrent neural net is described that learns a set of patterns in the presence of noise. The learning rule is of Hebbian type, and, if noise would be absent during the learning process, the resulting final values of the weights would correspond to the pseudo-inverse solution of the fixed point equation in question. For a non-vanishing noise parameter, an explicit expression for the expectation value of the weights is obtained. This result turns out to be unequal to the pseudo-inverse solution. Furthermore, the stability properties of the system are discussed.