Simultaneous embedding of multiple attractor manifolds in a recurrent neural network using constrained gradient optimization

TL;DR

This paper demonstrates how to embed multiple continuous attractor manifolds in a recurrent neural network using constrained gradient optimization to reduce interference and improve memory stability.

Contribution

It introduces a novel method for adjusting synaptic weights via a loss function to stabilize multiple attractors in a single RNN without losing capacity.

Findings

Significantly reduces interference between attractors

Enhances stability of stored continuous variables

Maintains network capacity while improving robustness

Abstract

The storage of continuous variables in working memory is hypothesized to be sustained in the brain by the dynamics of recurrent neural networks (RNNs) whose steady states form continuous manifolds. In some cases, it is thought that the synaptic connectivity supports multiple attractor manifolds, each mapped to a different context or task. For example, in hippocampal area CA3, positions in distinct environments are represented by distinct sets of population activity patterns, each forming a continuum. It has been argued that the embedding of multiple continuous attractors in a single RNN inevitably causes detrimental interference: quenched noise in the synaptic connectivity disrupts the continuity of each attractor, replacing it by a discrete set of steady states that can be conceptualized as lying on local minima of an abstract energy landscape. Consequently, population activity patterns exhibit systematic drifts towards one of these discrete minima, thereby degrading the stored memory over time. Here we show that it is possible to dramatically attenuate these detrimental interference effects by adjusting the synaptic weights. Synaptic weight adjustments are derived from a loss function that quantifies the roughness of the energy landscape along each of the embedded attractor manifolds. By minimizing this loss function, the stability of states can be dramatically improved, without compromising the capacity.