Sudden restructuring of memory representations in recurrent neural networks with repeated stimulus presentations

TL;DR

This paper shows that simple brain-like networks can experience sudden jumps in learning performance when repeatedly exposed to a stimulus, similar to human learning patterns.

Contribution

The study demonstrates that Amari-Hopfield networks exhibit discontinuous learning jumps with repeated stimulus exposure, mirroring human learning behavior.

Findings

Attractor basin size increases in discrete jumps rather than gradually with repeated stimulus exposure.

The distribution of these jumps follows a lognormal pattern, indicating a heavy-tailed distribution.

Newly acquired states form hierarchically branching structures with branch sizes following a power law distribution.

Abstract

While acquisition curves in human learning averaged at the group level display smooth, gradual changes in performance, individual learning curves across cognitive domains reveal sudden, discontinuous jumps in performance. Similar thresholding effects are a hallmark of a range of nonlinear systems which can be explored using simple, abstract models. Here, I investigate discontinuous changes in learning performance using Amari-Hopfield networks with Hebbian learning rules which are repeatedly exposed to a single stimulus. Simulations reveal that the attractor basin size for a target stimulus increases in discrete jumps rather than gradual changes with repeated stimulus exposure. The distribution of the size of these positive jumps in basin size is best approximated by a lognormal distribution, suggesting that the distribution is heavy-tailed. Examination of the transition graph structure for networks before and after basin size changes reveals that newly acquired states are often organized into hierarchically branching tree structures, and that the distribution of branch sizes is best approximated by a power law distribution. The findings suggest that even simple nonlinear network models of associative learning exhibit discontinuous changes in performance with repeated learning which mirror behavioral results observed in humans. Future work can investigate similar mechanisms in more biologically detailed network models, potentially offering insight into the network mechanisms of learning with repeated exposure or practice.

Figures