Iterative Neural Networks with Bounded Weights

TL;DR

This paper analyzes iterative neural networks in Hilbert spaces, establishing conditions for convergence to a unique fixed point and providing bounds on the fixed point's norm, with implications for network design.

Contribution

It introduces a mild weight condition ensuring convergence to a unique fixed point and derives bounds on the fixed point's norm, expanding theoretical understanding of iterative neural networks.

Findings

Networks converge to a unique fixed point under the weight condition.

A bound on the fixed point's norm is established.

The model cannot represent Hopfield networks under the given assumptions.

Abstract

A recent analysis of a model of iterative neural network in Hilbert spaces established fundamental properties of such networks, such as existence of the fixed points sets, convergence analysis, and Lipschitz continuity. Building on these results, we show that under a single mild condition on the weights of the network, one is guaranteed to obtain a neural network converging to its unique fixed point. We provide a bound on the norm of this fixed point in terms of norms of weights and biases of the network. We also show why this model of a feed-forward neural network is not able to accomodate Hopfield networks under our assumption.