Singular-Value-Decomposition Analysis of Associative Memory in a Neural Network

TL;DR

This paper investigates how singular value decomposition (SVD) influences associative memory performance in neural networks, revealing that the network's ability to retrieve original images depends on the input's entropy scaled by SVD, indicating a universal criterion.

Contribution

It introduces a novel analysis of associative memory performance using SVD and entropy scaling, demonstrating a universal criterion for successful image retrieval in neural networks.

Findings

Performance depends on input entropy exceeding a critical SVD-based threshold.

SVD provides an effective criterion for evaluating associative memory performance.

Scaling behavior observed suggests universality beyond theoretical physics.

Abstract

We evaluate performance of associative memory in a neural network by based on the singular value decomposition (SVD) of image data stored in the network. We consider the situation in which the original image and its highly coarse-grained one by SVD are stored in the network and the intermediate one is taken as an input. We find that the performance is characterized by the snapshot-entropy scaling inherent in the SVD: the network retrieves the original image when the entropy of the input image is larger than the critical value determined from the scaling. The result indicates efficiency of the SVD as a criterion of the performance and also indicates universality of the scaling for realistic problems beyond theoretical physics.