Memory retrieval dynamics and storage capacity of a modular network model of association cortex with featural decomposition

TL;DR

This paper models a modular neural network inspired by the primate cortex to analyze its memory retrieval dynamics and storage capacity, revealing near-optimal retrieval with minimal errors and examining the effects of synaptic scaling.

Contribution

It introduces a multimodular autoassociator model with finite connectivity and analyzes its capacity and dynamics, linking results to statistical mechanics and questioning the role of synaptic scaling.

Findings

Cued retrieval nearly saturates the upper bound with negligible spurious activation.

Storage capacity scales with the number of features per module following a combinatorial relationship.

Long-range synaptic scaling has minor impact on capacity and retrieval, challenging its presumed role in the neocortex.

Abstract

The primate heteromodal cortex presents an evident functional modularity at a mesoscopic level, with physiological and anatomical evidence pointing to it as likely substrate of long-term memory. In order to investigate some of its properties, a model of multimodular autoassociator is studied. Each of the many modules represents a neocortical functional ensemble of recurrently connected neurons and operates as a Hebbian autoassociator, storing a number of local features which it can recall upon cue. The global memory patterns are made of combinations of features sparsely distributed across the modules. Intermodular connections are modelled as a finite-connectivity random graph. Any pair of features in any respective pair of modules is allowed to be involved in several memory patterns; the coarse-grained modular network dynamics is defined in such a way as to overcome the consequent ambiguity of associations. Effects of long-range homeostatic synaptic scaling on network performance are also assessed. The dynamical process of cued retrieval almost saturates a natural upper bound while producing negligible spurious activation. The extent of finite-size effects on storage capacity is quantitatively evaluated. In the limit of infinite size, the functional relationship between storage capacity and number of features per module reduces to that which other authors found by methods from equilibrium statistical mechanics, which suggests that the origin of the functional form is of a combinatorial nature. In contrast with its apparent inevitability at intramodular level, long-range synaptic scaling results to be of minor relevance to both retrieval and storage capacity, casting doubt on its existence in the neocortex. A conjecture is also posited about how statistical fluctuation of connectivity across the network may underpin spontaneous emergence of semantic hierarchies through learning.