Localized activity profiles and storage capacity of rate-based autoassociative networks

TL;DR

This paper analytically investigates how spatially structured connectivity influences activity localization and storage capacity in rate-based autoassociative neural networks, revealing conditions for localized retrieval states and their impact on capacity.

Contribution

It provides a generic analytical framework for understanding localized activity and storage limits in rate-based autoassociative networks with spatially structured connectivity.

Findings

Localized retrieval states occur with short-range connectivity.

Spatially modulated states depend on maximum activity levels.

Localization reduces storage capacity by a factor of 2-3.

Abstract

We study analytically the effect of metrically structured connectivity on the behavior of autoassociative networks. We focus on three simple rate-based model neurons: threshold-linear, binary or smoothly saturating units. For a connectivity which is short range enough the threshold-linear network shows localized retrieval states. The saturating and binary models also exhibit spatially modulated retrieval states if the highest activity level that they can achieve is above the maximum activity of the units in the stored patterns. In the zero quenched noise limit, we derive an analytical formula for the critical value of the connectivity width below which one observes spatially non-uniform retrieval states. Localization reduces storage capacity, but only by a factor of 2~3. The approach that we present here is generic in the sense that there are no specific assumptions on the single unit input-output function nor on the exact connectivity structure.