The critical dimension of memory engrams and an optimal number of senses

TL;DR

This paper explores how many senses are best for memory and learning by analyzing a model of memory engrams and their interactions.

Contribution

The paper introduces a novel kinetic model of memory engrams and identifies a critical dimension for optimal memory capacity.

Findings

A critical dimension exists where the number of different engrams is maximal.

Higher receptivity to new stimuli reduces the sharpness of learned concepts.

The model shows a steady state with a fixed number of engrams covering part of the conceptual space.

Abstract

In this work, we analyse the fundamental question: how many senses are optimal for memory and learning. To answer this question, we introduce and analyse a novel kinetic model of memory engrams. The model, built on basic general principles and phenomenology, captures the engrams’ emergence and evolution driven by their interaction with external environment, learning, and forgetting. We derive the corresponding kinetic equation governing the dynamics and evolution of engrams over time. We then solve this equation analytically and numerically through Monte Carlo simulations. We observe the formation of a steady state with a steady number of different engrams covering a fraction of the conceptual space. We analyze the impact of the dimension of the conceptual space on the steady state and discover the existence of a critical dimension, at which the number of different engrams is maximal; we provide a theoretical explanation of this observation. If each feature is associated with a different sense, the critical dimension corresponds to an optimal number of senses for a system aiming at keeping the maximal number of different concepts in its memory. We also reveal an apparent tension between the system’s receptivity to new stimuli and concept sharpness—the higher the receptivity, the less sharp the learned concept becomes.

Figures

Schematic representation of the model. The blue intervals (or discs) schematically represent engrams. The large red arrows indicate the external stimuli; small black arrows show the direction of engram growth or shrinkage. When a stimulus strikes an engram, (a) and (b), the engram’s centers shift to the point of the strike (b) and (c). Without a strike an engram expands. Left panel illustrates 1d system, right panel—2d system.

The steady-state distribution for *f*(*l*) for one-dimensional engrams as a function of the engram size *l* for the parameters $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(l_{max},\alpha ,\tau ,l_{0},l_{*}) = (10,0.1,1,2,4)$$\end{document}$ (left) and $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \us

The number of distinct centers (NDC) over time for the model “learning from scratch” for fixed forgetting rate $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha = 0.002$$\end{document}$ with varied characteristic length $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\o

The number of distinct engram centers (NDC) over time *t* in 3d case, for parameters $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\alpha , A_{0},d) = (0.02,0.01,3)$$\end{document}$ (left) and $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{d

The parameters $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document}$ and $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0}$$\end{document}$ significantly affect the steady state distribution of engrams. In this 3

Left panel: The steady state number of different engram centers (NDC), as the function of *d*—the dimension of the conceptual space. A critical dimension $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c$$\end{document}$ exists where the number of different engrams is maximal. For $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbs

The number of engrams with different centers (NDC) over dimension *d* for different time, for two models with inhibition. Left panel—the model (i), when only engram with the minimum distance of its center from the hit point shrinks. Right panel—the model (ii), when only engram with the largest distance of its center from the hit point shrinks (see the text for more detail). The maximum area $\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{

Full text

Introduction

Learning and memory are arguably amongst the most enigmatic, inspiring, and widespread phenomena observed in living organisms. They co-exist and operate across a spectrum of scales including at the molecular level^1^, at the level of single neurons and their networks^2^, and extending to more abstract psychological models and theories^3^. Since 1904, when Richard Semon introduced the term “engram” to describe the neural substrate for storing memories^4^, much is known about potential physiological mechanisms supporting memory and memory engrams^5^—sparse ensembles of neurons across multiple brain regions—are viewed as viable hosts of memory^6^. This includes complementary but in no way contradictory evidence that individual neural cells, potentially within an engram, could already be associated with specific memory and concepts^7–10^. In the model presented below, this corresponds to “nested” engrams.

Although the engram concept has been introduced more than a century ago in^4^, the dynamics and evolution of engrams (and hence memories) after their emergence are still not fully understood. A very recent computational and experimental studies^11^ shed some light on how the neural composition and selectivity of single engrams change over a relatively short time. The authors identified specific mechanisms that may be responsible for the emergence of memory selectivity. Notwithstanding this and other works over more than 100 years, many fundamental questions about longer-time behavior of the building blocks of memory—“memory engrams”, remain open. These are:

• (i)Is the dimension of the concept space important, and if so then in what respect?
• (ii)Can we predict and model what happens with multiple memory engrams evolving and responding to stimulation over longer timescales?
• (iii)Do they retain encoding capabilities indefinitely, vanish, dissolve, or aggregate into larger entities encoding super-concepts?
• (iv)Does a self-supporting, steady state distribution of engram ensemble exist, and what would be the properties of such a distribution? In this paper, we present a pathway to answer these open foundational questions. We approach the problem by defining an appropriate mathematical and modelling framework enabling investigation of the long-term behavior of engrams. At the core of the approach is the appropriate reduction of the complexity of detailed biological processes describing the emergence and evolution of memory engrams and concepts to much simpler objects that are mathematically and computationally tractable at large spatial and temporal scales. This model must be based on a very general set of rules and reflect the most prominent features of the engrams, that is, how the engrams interact with the external world through stimuli. In particular, they must reflect the dynamic nature of engrams’ sensitivity to stimuli^11^. According to Ref.^12^ engrams are (i) activated during experience, (ii) undergo structural and functional modifications as a result and (iii) reactivated upon recall of the experience; all these features must be reflected in the model.

Early experimental studies in the field of engram research have focused on engram localization using various techniques such as lesion studies^13–15^, optogenetics^12,16–18^, transgenics^12,19^, pharmacogenetics^12^, immediate early gene (IEG)^20,21^ etc.. These types of studies have paved the way for constructing computational methods such as attractor networks^22,23^, Hopfield networks^24^ and spiking Hebbian networks^25,26^ to simulate memory encoding, consolidation and retrieval. The current state of engram research is the further development of these models as attractor networks focuses on simulating memory stored as stable patterns^27^, spiking Hebbian networks centers on studying the relation between associative memory and sequence generation^26^ and Hopfield networks draws attention to associative memory and pattern recognition where memory is retrieved from information that is partial or corrupted^24^. Despite theoretical and experimental progress in these areas, a sophisticated model which captures the temporal evolution of engrams over time is needed. We formulate kinetic models of memory engrams which describe the size distribution of engrams in d-dimensional space. Our approach in solving the equations is employing Monte-Carlo simulations. Not only does our model incorporate the activation/reactivation of engrams through stimuli and reflect changes in engram formation over time, the observed result where higher receptivity of engrams results in learned concepts becoming less sharp aligns with the effects of high receptivity on attractor and spiking Hebbian networks. That is, high receptivity on attractor and spiking Hebbian networks respectively causes convergence to a less distinct attractor state which reduces sharpness and more diffuse synaptic changes which reduces the memory representation’s sharpness. With appropriate setting, our model can simulate manipulation of engrams implemented in loss of function studies (engram impairment leading to the impairment of memory retrieval)^28^, gain of function studies (artificial activation of engrams for memory retrieval)^28^, mimicry studies (artificial introduction of engrams of unreal experience)^28,29^ and studies whereby memory maintenance is showcased by replacing some engrams while others remain stable.

In our model, we assume that a stimulus is defined by a finite set of features, and each feature is encoded by a real number. A stimulus therefore is an element of a linear vector space. We assume that stimuli are comparable with each other in the sense that there is a metric in the space of stimuli enabling us to tell how far one stimulus is away from the other. This property is needed to model how the stimuli are produced in the external environment.

A concept or an engram is an object that can “sense” a stimulus, but its sensitivity is dynamic. Not all stimuli can be sensed or recalled by a single engram. Moreover, the set of all potential stimuli, a single engram can sense or respond to, may change over time. To represent these properties in the simplest way we model engrams as sets in the d-dimensional space (associated with a number of stimuli of different nature), whose composition changes over time. The sets representing engrams can grow or shrink over time capturing the engrams’ dynamic sensitivity. The smaller the set is the higher the engram’s sensitivity.

The space in which all engrams are defined is referred as an ambient space and the space in which they dynamically evolve will be called a conceptual space or the space of encoded concepts stored in memory. The concepts encoded through engrams are not “rigid”—they can partly or completely overlap in the conceptual space, or form nested engrams, when one engram encompasses another one.

To capture stimuli interaction with engrams we follow the recent findings reported in^11^. When a stimulus is sensed by an engram, the engram responds by increasing its sensitivity to that stimulus. In our model, this is equivalent to the engram retaining those elements from its own composition that are sufficiently close to the stimulus and releasing elements that are deemed too far. An activating stimulus may be considered as a “hit” at some point in the conceptual space. That is, it corresponds to an “impression” associated with the set of particular features \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(a,b,c,\ldots )$$\end{document} of an object. If a stimulus “hits” some engram, then an interaction between the stimulus and the engram takes place reflecting the known wisdom—“I see what I know”. If the stimulus is not in any of the existing engrams in the conceptual space, then two alternatives are possible. In the first alternative, the stimulus gets lost without any impact on the conceptual space. In the second alternative, a new engram emerges with the stimulus being among its elements.

These two alternatives give rise to two cases of long-term learning we analyse in our work: “confirming learning” and “learning from scratch”. In the former case, we analyse an already established system, in which a set of engrams/concepts is readily formed and stimuli not hitting a concept are permanently lost. In this regime, we explore the evolution of the system of concepts in response to stimuli. Namely, we study the formation and properties of a steady state. In the latter case, of “learning from scratch”, we analyse the formation of the system of concepts—we assume that a new concept emerges when a few stimuli hit some region in the conceptual space (CS) during a short period of time.

In what follows we present a kinetic model capturing the above description of the evolution and dynamics of engrams and provide both analytical and numerical analysis of its relevant dynamics properties, including the existence and description of steady-states. We also explore how properties of the steady-state depend on the dimension of the conceptual space and whether some dimensions with extreme properties of the conceptual space exist.

The rest of the paper is organized as follows. In Section 2 we present the main results of our work. We start with a one-dimensional model and its analysis, including an analytical solution of the engrams’ steady state and its properties. We then continue with d-dimensional models and reveal the existence of a critical dimension of the engrams’ conceptual space at which the number of distinct engrams is maximized. Section 3 describes the methods we used in our exploration, and Section 4 contains a short discussion and a summary of our findings. Additional technical details, including figures (Fig. S1–S5) and analytical derivation of the maximal number of engrams with distinct centres in different settings are provided in Supplementary Materials.

Results

Kinetics of engrams in one-dimensional conceptual space

Confirming learning

To illustrate our approach, we start with a toy-model—a one-dimensional model of conceptual space. Since there are no prior reasons for the existence of special ’boundary domains’ in the conceptual space, we assume periodic boundary conditions. In particular, we suppose that the conceptual space is reminiscent of a circle or, more generally, a sphere in higher dimensions. This choice is particularly natural for higher dimensions due to the well-know shell concentration^30^ and stochastic separation^31,32^ effects.

In the simplest setting, engrams are one-dimensional segments of different lengths located on a circle in two-dimensional ambient space. Geometrically, the system comprises n segments of length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l \in (0,l_{max}]$$\end{document} . Each segment \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{i}$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(i = 1,2,...,n)$$\end{document} is randomly distributed on a circle whose length is L and, depending on the value of L and/or the total number of segments n, may overlap with other segments \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{j}$$\end{document} ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$i \ne j$$\end{document} ).

Stimuli, which will be referred to as the Catching Shots (CS), continuously ’hit’ the concept space at random, at the rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1/\tau$$\end{document} . They may strike points on the concept space where segments are present or they hit elsewhere. Each catching shot hits the concept space at the average density \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1/l_*$$\end{document} , with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_*=L$$\end{document} , so that, at the time interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Delta t$$\end{document} , any point in the interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Delta x$$\end{document} would be hit with the probability \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Delta x \Delta t / l_{*} \tau$$\end{document} . The shot instantly “evaporates” when it hits a point where no segments exist, that is, such hits make no impact on the interaction between future CS and engrams.

The moment a catching shot strikes a segment, the center of the hit segment moves to the point of the catching shot. Next, the length of the displaced segment reduces to size l with the probability

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P_\textrm{foc}(l) dl= l_{0}^{-1} e^{-l/l_{0}}dl \end{aligned}$$\end{document}

where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\int _{0}^{\infty } P_\textrm{foc}(l) dl = 1$$\end{document} . This reflects the engram property of sharpening in response to the CS and its affinity to the very last stimulus. That is, the conceptual segment (engram) becomes more focused. Note that Eq. (1) defines the *probability * for a segment length to be equal to l after a strike. In general, l may exceed the current length of the hit segment \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l^{\prime }$$\end{document} . In this case ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l>l^{\prime }$$\end{document} ) a more adequate model would read \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_\textrm{foc}(l|l^{\prime })=e^{-l/l_0}\Theta (l^{\prime }-l)(l_0-l_0e^{-l^{\prime }/l_0})^{-1}$$\end{document} , preventing a length increase after a strike. However, if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} is small, that is, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0^2 \ll \alpha \tau l_*$$\end{document} , the probability that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l>l^{\prime }$$\end{document} would be insignificant and such cases may be neglected (see the definition of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} below and the Supplementary Material).

In what follows we will assume that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} is small and exploit more simple and tractable model, Eq. (1), which is the most simple probabilistic model with an explicit characteristic length. The choice of the exponential density in (1) can additionally be motivated by the maximal entropy principle: it corresponds to the density with the highest entropy, or uncertainty, whose first moment equals \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} . This enables modeling ”imprecise” systems in which the system’s expected focusing capability remains constant but the segment size distribution after hits exhibits the highest degree of uncertainty. Further simplifications lead to the case where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_\textrm{foc}(l) dl = \delta (l-l_0)dl$$\end{document} , with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\delta$$\end{document} being the Dirac delta function. In the latter setting, the segments always contract to the same length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} which is hardly realistic.

After shrinkage, the length of the hit segment increases with the rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha e^{-l/l_{max}}$$\end{document} where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha >0$$\end{document} is a parameter controlling the growth rate of the segment and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{max}$$\end{document} is the maximum length a segment can reach (the growth stops when the length reaches \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{max}$$\end{document} ). This reflects engrams’ tendency or predisposition to lose sharpness over time in absence of external stimuli. Segments that have not been hit retain their centres but their size grows with the same length-dependent rate as follows:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{dl}{dt}&= \alpha e^{-l/l_{max}} \Theta (l_{max}-l), \end{aligned}$$\end{document}

where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta (x)$$\end{document} is the standard unit Heaviside step-function. Empirical evidence^33^ suggests that forgetting processes could be effectively modelled by sums of exponents with negative rates. A typical pattern is a sharp initial decay followed by a much slower relaxation to the baseline. In our model, we accommodate these phenomenological observations by slowing the growth rate of the segment (the forgetting rate) as the size of the engram grows: the more fuzzy an idea (concept) becomes the slower it loses its sharpness. Instead of employing higher-order linear dynamics to implement multiple exponents, we opted for a single nonlinear ordinary differential equation (2) capable of mimicking a variable rate of decay. Its solutions converge to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{max}$$\end{document} in finite time, indicating that in our model the maximally plausible loss of concept sharpness in the absence of stimuli occurs in finite time too. In principle, one could also consider a more simplified version of the model, with linear forgetting time: \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$dl/dt=\alpha \,\Theta (l_{max}-l)$$\end{document} . Although offering a computational advantage and simplicity, the simplified model cannot reproduce the slow-fast forgetting dynamics observed in the literature.

The general combined model of the system reflects the fact that once a physiological support, or a material carrier, of a concept (and an associated engram) is formed it can evolve and transform but, being the neural substrate for storing memories, can hardly completely vanish in the absence of adverse physiological changes (traumas, illness, etc.)^28^. This is consistent with experimental in-vivo findings revealing that memory formation in developed mice primarily recruits preexisting neural assemblies, rather than forms new neural assemblies^34^. Figure 1 schematically illustrates the model.Fig. 1. Schematic representation of the model. The blue intervals (or discs) schematically represent engrams. The large red arrows indicate the external stimuli; small black arrows show the direction of engram growth or shrinkage. When a stimulus strikes an engram, (a) and (b), the engram’s centers shift to the point of the strike (b) and (c). Without a strike an engram expands. Left panel illustrates 1d system, right panel—2d system.

Let the number density of segments of the length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l \in [l,l+dl]$$\end{document} , whose centers are located at point x at time instant t, be characterized by the distribution function f(x, l, t), which is normalized to the total number of intervals n as

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \int _0^{L}dx\int _0^{l_{max}} f(x,l,t)dl =n. \end{aligned}$$\end{document}

Assuming that distribution of the interval is space uniform, that is, the function does not depend on x, we can write for the increment of this function during the time interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$[t,t + \Delta t]$$\end{document} ,

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} {[}f(x,l,t + \Delta t) - f(x,l,t)]dl&= - \frac{l \Delta t}{\tau l_{*}} f(x,l,t) dl - \Delta t dl \frac{\partial }{\partial l} \bigg (\frac{\partial l}{\partial t}\bigg ) f(x,l,t) \nonumber \\&\hspace{1cm} + P_\textrm{foc}(l)dl \int _{0}^{l_{max}} f(x,l',t) \frac{l' \Delta t}{l_{*}\tau } dl'. \end{aligned}$$\end{document}

The first and second term on the rhs of Eq. (4) respectively give the number of segments of length l which disappear during the time interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Delta t$$\end{document} due to the shots and the growth of the intervals. The third term gives the appearance of the segment of length l due to the intervals of length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l' \in [0,l_{max}]$$\end{document} being hit into. Segments of size l emerge (from the hit intervals of the length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l'$$\end{document} ) with the rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_\textrm{foc}(l)dl=e^{-\frac{l}{l_{0}}}dl/l_0$$\end{document} .

For small \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Delta t$$\end{document} , the expansion of the lhs of Eq. (4) yields the equation, which may be called “engram kinetic equation”:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{\partial f(l,t)}{\partial t}&= - \frac{l}{\tau l_{*}} f(l,t) - \frac{\partial }{\partial l} \bigg (\frac{dl}{d t}\bigg ) f(l,t) + P_\textrm{foc}(l) \int _{0}^{l_{max}} f(l',t) \frac{l' }{l_{*}\tau } dl'. \end{aligned}$$\end{document}

Here we take into account the assumption that the density is uniform and does not depend on x. The n-dimensional generalization of the above equation is presented below. Note that during evolution two or more intervals may intersect and, consequently, may be hit by the same CS landing in the domain in their intersection. In this case, after the hit, the intervals would share the same centre. However, their sizes will be different as being randomly chosen from the appropriate Poisson distribution. Hence they do not merge because they have different size. An alterative case is discussed in Section 2.2.4 (Kinetics of engrams with inhibition).

Using Eqs. (1) and (2) we recast Eq. (5) into the form,

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{\partial f(l,t)}{\partial t}&= - \frac{l}{\tau l_{*}} f(l,t) - \alpha \frac{\partial }{\partial l} \bigg [e^{-l/l_{max}} \Theta (l_{max}-l) f(l,t)\bigg ] + \frac{e^{-\frac{l}{l_{0}}}}{ \tau l_{0}} F(t) \end{aligned}$$\end{document}

where

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} F(t) = \int _{0}^{l_{max}} f(l',t) \frac{l' }{l_{*}} dl'. \end{aligned}$$\end{document}

We notice that the average interval length reads,

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\langle l \rangle =\frac{\int _{0}^{l_{max}} f(x,l',t) l'dl'}{\int _{0}^{l_{max}} f(x,l',t) dl'} =\frac{F(t) l^*}{n/L},$$\end{document}

where we use the normalization condition (3) for f(x, l, t); this yields for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l^*=L$$\end{document} ,

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F(t)= \rho \frac{\langle l \rangle }{L},$$\end{document}

with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\rho =n/L$$\end{document} being the density of the intervals.

The stationary solution f(l, t) of the integro-differential equation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(5)$$\end{document} is independent of t along with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$F(t)=F$$\end{document} (a time independent constant). For a space-uniform system f does not depend on x. Introducing new variable \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$z = l/l_{max}$$\end{document} we write the stationary form of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(5)$$\end{document} for the function \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tilde{f}(z) \equiv f(z\cdot l_{max})$$\end{document} as

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{\partial }{\partial z} \big (e^{-z} \tilde{f}(z) \big ) = -B_{1}z\tilde{f}(z) + B_{2}Fe^{-az} \end{aligned}$$\end{document}

or

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{\partial \tilde{f}}{\partial z} = -B_{1}e^{z} z \tilde{f}(z) + B_{2}Fe^{(1-a)z} + \tilde{f}(z) \end{aligned}$$\end{document}

with

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} B_{1} = \frac{l_{max}^{2}}{l_{*} \alpha \tau }, \hspace{0.5cm} B_{2} = \frac{l_{max}}{l_{0} \alpha \tau }, \hspace{0.5cm} \text {and} \hspace{0.5cm} a = \frac{l_{max}}{l_{0}}. \end{aligned}$$\end{document}

The solution to the steady state equation \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(8)$$\end{document} yields

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \tilde{f}(z) = B_{2}Fe^{-B_{1}(z-1)e^{z} + z} \int _{1}^{e^{z}} w^{-(1+a)} e^{B_{1}(\log w - 1)w} dw \end{aligned}$$\end{document}

for condition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f(0) = 0$$\end{document} . Figure 2 shows the numerical solution of (6), (7) at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t = 1000$$\end{document} (when a steady-state is achieved) with the initial condition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f(0,l) = 1$$\end{document} for the parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(l_{max},\alpha ,\tau ,l_{0},l_{*}) = (10,0.1,1,2,4)$$\end{document} (left) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(l_{max},\alpha ,\tau ,l_{0},l_{*}) = (10,1,1,2,4)$$\end{document} (right).Fig. 2. The steady-state distribution for f(l) for one-dimensional engrams as a function of the engram size l for the parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(l_{max},\alpha ,\tau ,l_{0},l_{*}) = (10,0.1,1,2,4)$$\end{document} (left) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(l_{max},\alpha ,\tau ,l_{0},l_{*}) = (10,1,1,2,4)$$\end{document} (right) with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f(0) = 0$$\end{document} . f(l) is a non-monotonic function of l which approaches zero for large l. The numerical solution of Eq. (6), (7) is indistinguishable from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f(l/l_{max})$$\end{document} given by Eq. (9), with self-consistently found F.

As can be seen in the figure, the distribution of the conceptual segments is peaked around the characteristic length corresponding to the size of the segment, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} , immediately after an “impression strike”; the distribution sensitively depends on the learning (hitting) rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1/\tau$$\end{document} and the forgetting rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} . This fact may be interpreted as follows: the stationary distribution is determined by the trade-off between the focusing of the concepts (which helps to be more specific in classifying the object of the external world) and the ability to better react to the external stimuli (which is larger when “the conceptual segments” have larger size). We expect that during evolution nature “chooses” appropriate values of parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau$$\end{document} balancing the focusing and forgetting, depending on the learning (hitting) rate. Noteworthy, the oversimplified model with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_\textrm{foc}(l)=\delta (l-l_0)$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$dl/dt= \alpha \, \Theta (l_{max}-l)$$\end{document} also yields a peaked at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l=l_0$$\end{document} distribution, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$f(l)= (F/\alpha \tau l_0) \exp [-(l^2-l_0^2)/(2\alpha \tau l_*)] \Theta (l-l_0)$$\end{document} , with the singularity at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l=l_0$$\end{document} , which makes this model mathematically less convenient, see the Supplementary Material (SM).

Learning from scratch

Here we use the following model for the learning from scratch: A new concept, i.e. engram, appears^28^, if q successive stimuli hit during a time interval \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau$$\end{document} an interval of length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_s$$\end{document} of the conceptual space, which is not occupied by other engrams. In this case a new engram—a segment of size \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_s$$\end{document} , centered at the point of the last hit, emerges.

We consider a system with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N = 1000$$\end{document} segments randomly distributed onto a line, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$L = 250$$\end{document} , with a minimum and maximum length of 0.1 and 20 respectively. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{0}$$\end{document} are varied independently to investigate the number of distinct centers (NDC) over time after a new segment appears when the same empty region is successively hit \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$q=3$$\end{document} times. We observe the behaviour of NDC reaching a steady state for each forgetting rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and characteristic length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{0}$$\end{document} .Fig. 3. The number of distinct centers (NDC) over time for the model “learning from scratch” for fixed forgetting rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha = 0.002$$\end{document} with varied characteristic length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{0}$$\end{document} .

NDC initially decreases with time as overlapping hit segments group together to share midpoints. After a significant drop in NDC, it quickly reaches a steady state (see Fig. 3). The probability of introducing a new segment from scratch is much higher for parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{0}$$\end{document} where NDC is very small. This high probability results to the addition of new segments which will eventually overlap with other segments and group together when hit. The addition of new segments can also lead to center sharing segments breaking apart and forming new centers with the added segments. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} controls the growth of segments after the size of the hit segments are changed. For sufficiently small \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} , the number of segments introduced into the system grows as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} increases. For small \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and varied \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{0}$$\end{document} , NDC decreases with increasing \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} . At large \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} , NDC is at its minimal value. In addition to that, the number of empty regions reduces, as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} is large, which in turn reduces the number of segments introduced to the system via successive hits. For larger \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and varied \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_{0}$$\end{document} , the value of NDC is small and similar for each \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} however the number of segments that are introduced decreases.

These results clearly demonstrate an important property of engrams – for large forgetting rates the system quickly attains a stationary state at which its “memory” capacity (quantified by the NDC) drops and stays relatively small over time. Interestingly, the steady state memory capacity fluctuates around a quantity (200-300 and 50-100 in terms of NDC for our setup for larger \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} i.e. \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha = 0.02$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha = 0.2$$\end{document} respectively) which is seemingly independent of the characteristic length \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$l_0$$\end{document} . Hence, the proposed model of the engrams predicts the existence of an intrinsic steady state, which is an important property of learning. Moreover the model with the learning from scratch demonstrates qualitatively similar behavior, as the basic model. For more plots characterizing the kinetics of learning from scratch, for different kinetic parameters see SM, Fig. S1.

Kinetics of engrams in d-dimensional ambient space

General setup

Now we consider kinetics of learning in multidimensional systems, in d-dimensional ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d> 2$$\end{document} ) ambient space. This implies that conceptual space is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d-1$$\end{document} dimensional, represented by the surface of an appropriate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S^{d-1}$$\end{document} sphere. In order to compare engrams’ properties for different dimensions d, we consider engrams’ dynamics in normalized conceptual space. That is if \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(a,b,c,\ldots )$$\end{document} are features of the stimuli then suppose that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\sqrt{a^2+b^2+c^2 +...}=1$$\end{document} . Based on the nature of the system, we require the following properties of the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d-1$$\end{document} -dimensional normalized manifold: (i) it should not have any boundaries, as the boundaries correspond to a sub-manifold with special properties and (ii) the manifold should be topologically smooth. The surface of a d-dimensional ball corresponds to the \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d-1$$\end{document} -dimensional manifold with the requested properties. Note that we tacitly assume the homogeneity of the concept space, although concepts themselves may cluster (semantic clustering), possess learning history, etc. However, in a lack of a sufficient knowledge about such effects, we apply a simple idealized model of uniform conceptual space—a manifold in an abstract space, where the objects—concepts may be distributed non-uniformly. The same assumption of homogeneity (and for the same reasons) is applied for the space distribution of the hits.

Let us now consider this d-dimensional model describing the growth/shrinkage of segments distributed on the surface \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S^{d-1}$$\end{document} of the unit ball \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B_{d}(0,1)$$\end{document} centered at the origin in more detail. The model consists of n spherical caps (corresponding to segments in our previous model) \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{r_{i},\vec {c}_{i}} \subset S^{d-1}$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(i = 1,2,...,n)$$\end{document} with distinct radii \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$r_{i}$$\end{document} , centers \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {c}_{i} = (x_{1}, x_{2}, x_{3},..., x_{d})$$\end{document} ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$|\vec {c}_{i}| = 1$$\end{document} ) and area \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{i} \in (0,A_{max}]$$\end{document} . The caps \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{r_{i},\vec {c}_{i}}$$\end{document} are randomly distributed on the surface \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S^{d-1} \subset B_{d}(0,1)$$\end{document} . Their area is:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} A_{i} = \text {area}_{d-1}(C_{r_{i},\vec {c}_{i}})&= \frac{2 \pi ^{\frac{d-1}{2}}}{\Gamma (\frac{d-1}{2})} \int _{0}^{\hat{r}_{i}} \sin ^{d-2} \theta \; d \theta = \frac{\pi ^{\frac{d-1}{2}}}{\Gamma (\frac{d-1}{2})} B\bigg (\sin ^{2}(\hat{r}_{i}),\frac{d-1}{2},\frac{1}{2}\bigg ), \end{aligned}$$\end{document}

where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\hat{r}_{i}=r_i/R =r_i$$\end{document} is the polar angle of the cap, with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$R=1$$\end{document} being the radius of the unit ball \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B_{d}(0,1)$$\end{document} , and B(x, a, b) is the incomplete beta function expressed as

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} B(x,a,b) = \int _{0}^{x} t^{a-1} (1-t)^{b-1} dt. \end{aligned}$$\end{document}

For instance, in the dimension \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d = 3$$\end{document} , the cap \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{r_{i},\vec {c}_{i}} \subset S^{2}$$\end{document} centered at the point \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {c}_{i} = (x_{1},x_{2},x_{3})$$\end{document} and located on the surface of the unit sphere \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B_{3}(0,1)$$\end{document} will have the following area:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} A_{i} = 2 \pi (1 - \cos (\hat{r}_{i})). \end{aligned}$$\end{document}

The maximum area the spherical cap is allowed to grow to is denoted by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{max}$$\end{document} .

As we have seen in the previous section, the confirming learning and learning from scratch both possess steady-state, whose properties do not differ much. Therefore, we will focus below on the confirming learning. Similar to the case of 1 dimensional model studied which we discussed and analyzed earlier, catching shots successively drop onto \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S^{d-1}$$\end{document} , at the rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1/\tau$$\end{document} , striking points \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {X}_{p} = (x_{1}, x_{2},..., x_{d})$$\end{document} . They may hit none, one, or several spherical caps at a time. The caps are allowed to overlap. As before, the shots instantly evaporate when none of the spherical caps get hit. This corresponds to a state in the model when the stimulus is completely missed and left “no impression” on the existing engrams. When a catching shot falls inside \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$C_{r_{i},\vec {c}_{i}}$$\end{document} , the center \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {c}_{i}$$\end{document} of the hit spherical cap moves to the point of the catching shot \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {X}_{p}$$\end{document} . The area of the displaced cap then reduces in accordance with the Poisson distribution with the average shrinkage size of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0}$$\end{document} . If a cap is not hit, then its area \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_i$$\end{document} grows at the rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha e^{-A_i/A_{\max }}\Theta (A_{\max }-A_i)$$\end{document} .

The engram kinetic equation for d-dimensional case is a straightforward generalization of the respective one-dimensional equation:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{\partial f(A,t)}{\partial t}&= - \frac{A}{\tau A_{*}} f(A,t) - \frac{\partial }{\partial A} \bigg (\frac{dA}{d t}\bigg ) f(A,t) + P_\textrm{foc}(A) \int _{0}^{A_{max}} f(A',t) \frac{A' }{A_{*}\tau } dA', \end{aligned}$$\end{document}

where A is the size ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d-1$$\end{document} -dimensional area of an engram), \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_*$$\end{document} – the total area of the conceptual space. We assume that the forgetting (that is, the increase of an engram area) and focusing after a conceptual hit (that is, an engram shrinking after a hit) occur in a similar way as in one-dimensional case.

Hence, d-dimensional kinetic equation for engrams may be derived in the same way as for one-dimensional model:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{\partial f(A,t)}{\partial t}&= - \frac{A}{\tau A_{*}} f(A,t) - \frac{\partial }{\partial A} \bigg (\alpha e^{-A/A_\textrm{max}} \Theta (A_\textrm{max}-A)\bigg ) f(A,t) + A_0^{-1}e^{-A/A_0} \int _{0}^{A_{max}} f(A',t) \frac{A' }{A_{*}\tau } dA'. \end{aligned}$$\end{document}

Here A denotes of the area (size) of the engram, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{*}=2 \pi ^{d/2}/\Gamma (d/2)$$\end{document} is the surface area of the corresponding unit sphere in d dimensional ambient space, and

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} P_\textrm{foc}(A)= & A_0^{-1}e^{-A/A_0} \end{aligned}$$\end{document} \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{dA}{dt}= & \alpha e^{-A/A_\textrm{max}} \Theta (A_\textrm{max}-A). \end{aligned}$$\end{document}

For the simplified model discussed in Sec. 2.1, and which is more rigid in terms of phenomenological plausibility and mathematically less convenient due to singularities at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A=A_0$$\end{document} , one could choose \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$P_\textrm{foc}(A)= \delta (A-A_0)$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$dA/dt = \alpha A \Theta (A_\textrm{max}-A)$$\end{document} . In the above equations, Eqs. (13), (14), we assume that the conceptual space is uniform, so that f(A, t) does not depend on the coordinates \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\vec {X}=(x_1,x_2, \ldots , x_d)$$\end{document} . The structure of Eqs. (12)–(14) is identical to Eq. (6). Therefore, the overall description of the steady-state solution of these equations coincides with Eq. (9), up to an appropriate change of notation. Note that kinetic equations (6) and (12)–(14) are essentially mean-field equations, which lack fluctuations.

To study kinetic properties of the model, reflecting fluctuations, we conducted relevant Monte Carlo simulations, associated with the process (see Methods and Supplementary Material (SM) for detail). For each dimension, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n = 1000$$\end{document} spherical caps are randomly distributed on the surface \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$S^{d-1}$$\end{document} of the unit ball \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$B_{d}(0,1)$$\end{document} . Two kinetic parameters are investigated: the growth rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} (the forgetting rate) and counterpart—the expected focusing/shrinkage size \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0}$$\end{document} (the “focusing” parameter).

The aim is to investigate the impact of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0}$$\end{document} on the number of distinct centers (NDC) at large terminal times t corresponding to the end time of the simulation. Recall that our model enables multiple engrams to collide and share the same centres as a result of the collision. Exploring the entire two-dimensional parametric space is computationally challenging for large t. Therefore, we assumed that the forgetting rate and the focusing are correlated. In particular, we confined our exploration to the case when smaller forgetting rates imply that the focusing is sharper and vice-versa. We explored the evolution of engrams at the following values of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_0$$\end{document} : \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha = 2 \times 10^{-p}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0} = 1 \times 10^{-p}$$\end{document} , where p is a positive constant which we varied from one experiment to another. This enabled us to assess the system’s behavior across a broad range of temporal and spatial scales. (In practice, since \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{max}(d)$$\end{document} decreases with d and becomes small, we need to use the values of p satisfying the condition \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0} < A_{max}(d)$$\end{document} , which limits p from below.)

The rate of arrival of the catching shots was set to one. This fixed the time unit of the kinetic process. By varying the values of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} we could probe different engram evolution regimes. This included the regime when spherical caps grow rapidly after being hit but the frequency of arriving shots is low, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha \gg l_0/\tau$$\end{document} . This regime corresponds to a system of “fast forgetting”. We also looked at the regime when the growth of caps is slow relative to the frequency of arriving shots, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha \ll l_0/\tau$$\end{document} , which could be interpreted as a system with “slow forgetting”. The value of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{max}$$\end{document} was set to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pi ^{d/2}/2\Gamma (d/2)$$\end{document} (quarter of the size of the conceptual space) in all modeling scenarios.

Simulation results for d-dimensional ambient space

In our MC simulations we explored the evolution of engrams for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d\ge 3$$\end{document} . The main macroscopic feature we tracked in our simulations was the dynamics of the number of distinct centers over time and the number of distinct centers at the end of the temporal evolution of the system as a function of dimension d.

We observe that the number of distinct engram centers (NDC), after a short transient period, converges to a vicinity of some steady state value and keeps fluctuating around this state for the remaining time of the simulation. This initial increase/decrease in the number of distinct engram centers is mostly dependent on the ratio of the initial number of distinct centers \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n_{d}$$\end{document} and the total number of engrams n. When the ratio \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n_{d}/n$$\end{document} is small at the start of the evolution, engrams sharing a common centre but whose sizes are different break apart over time. This results in an increase in the number of distinct engram centers over time before the steady state is reached. The opposite occurs when \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n_{d}/n$$\end{document} is sufficiently large. NDC decreases as engrams continuously group together over time. Numerical values of the steady state did not visibly change for different ratios \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n_{d}/n$$\end{document} (as seen in Fig. 4). That is, the steady state value of engrams with different centers was largely independent on the initial number of engrams with distinct centers. When the areas of the hit engrams shrink to the same value under the Poisson distribution, engrams in the same groupings do not break apart.Fig. 4. The number of distinct engram centers (NDC) over time t in 3d case, for parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\alpha , A_{0},d) = (0.02,0.01,3)$$\end{document} (left) and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\alpha , A_{0},d) = (0.2,0.1,3)$$\end{document} (right). Different initial number of distinct engram centers \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n_{d}$$\end{document} are investigated with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n = 1000$$\end{document} total number of engrams. The ratio \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$n_{d}/n$$\end{document} determines whether NDC monotonically/non-monotonically increases/decreases over time. Eventually, the number of engrams with different centers in the conceptual space (NDC) over time fluctuates around the same steady state value for any initial number of distinct engram centers.

Figure (5) visualizes the model engram distribution in the steady state depending on the kinetic parameters of the model, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\alpha ,A_{0})$$\end{document} . When \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0}$$\end{document} are both large, there is a small number of independent engrams with different centers and a large number of engrams sharing the same center—they form a nested system of engrams. Contrary, when \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0}$$\end{document} are both small, the model predicts a large number of small independent engrams. Hence one can clearly see two opposite tendencies—large forgetting rate (large \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} ), with less focusing rate (large \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_0$$\end{document} ) result in larger covering of the conceptual space by the engrams, but they are less focused and many engrams share the same center (i.e. the dominance of nested engrams is observed). In contrast, for small forgetting rate and sharp focusing, all engrams are more focused and specialized (only a few share the center), but the total conceptual space is significantly less covered by the engrams. Possibly, there exist optimal values of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\alpha ,A_{0})$$\end{document} which correspond to the trade-off between these two tendencies.Fig. 5. The parameters \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{0}$$\end{document} significantly affect the steady state distribution of engrams. In this 3d example, we observe a large number of small-size individual engrams when \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\alpha ,A_{0})$$\end{document} are small (left) and a small number of large individual engrams when \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(\alpha ,A_{0})$$\end{document} are sufficiently large (right). The former case a large part of the conceptual space remains uncovered by the engrams, while in the latter case it is much more covered. This figure was produced using Matlab R2022b.

Critical dimension for evolving engrams

Next, we explore the dependence of the number of separate engrams in the steady state on the dimension of the ambient space d; the results are presented in Fig. 6. Surprisingly, this dependence is not monotonous. Namely, the number of different engram centers initially increases with d, reaching a maximum and then decreases, see Fig. 6 (left). Interestingly, the critical dimension slightly decreases when the forgetting rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} increases. That is, the smaller the forgetting rate the larger the critical dimension. At the same time, we observed that for larger values of the forgetting rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} critical dimension \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_s$$\end{document} saturates around \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c=7$$\end{document} for \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha > 10^{-4}$$\end{document} , see Fig. 6 (right). This suggests that the region around \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c=7$$\end{document} may correspond to an optimal dimension in terms of the capacity of the conceptual space.

Below we present a qualitative theoretical explanation for the existence of the critical dimension. Let \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{ tot}(t)$$\end{document} be the total area occupied by N(t) engrams with different centers at some time instance t in d-dimensional ambient space. Then the average area occupied by a single engram is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{ tot}(t)/N(t)$$\end{document} . Suppose that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{ tot}(t)/N(t)>A_0$$\end{document} . When an engram is hit by a catching shot, its area, on average, shrinks to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_0$$\end{document} (see see Eq. (13)). Let m be the average number of (overlapping) engrams affected by a single hit. Then each hit, on average, reduces the total area occupied by engrams by \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$m(A_{ tot}(t)/N(t) - A_0)$$\end{document} . Given that the average hit rate can be estimated as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1/\tau \cdot A_{tot}/A^*$$\end{document} , where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$1/\tau$$\end{document} is the relative frequency of shots and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A^*$$\end{document} is the area of the entire concept space, the overall reduction of the area covered by engrams due to the system’s exposure to stimuli can be estimated as:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\frac{A_{tot}(t)}{\tau A^*}m(A_{ tot}(t)/N(t) - A_0).$$\end{document}

At the same time, the growth rate of the total area occupied by N engrams with different centers, may be approximated by the growth rate of a single engram of the average size \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tot}/N$$\end{document} , which is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha e^{-A_{tot}/(N\, A_{max})}$$\end{document} , multiplied by the number of engrams N, yielding

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N \alpha e^{-A_{tot}/(N\, A_{max})},$$\end{document}

where we neglect that engrams may overlap and skip the factor \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\Theta (A_{max}-A_{tot}/N)$$\end{document} , see the discussion below. Hence, one can write the following mean-field kinetic equation governing the dynamics of the total area occupied by engrams:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \frac{d}{dt} A_{ tot} \simeq \alpha N \,e^{-A_{tot}/(N\,A_\textrm{max})} - \frac{A_{ tot}}{\tau A_*} \left[ m \left( \frac{A_{ tot}}{N} - A_0 \right) \right] . \end{aligned}$$\end{document}

For a steady state, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$dA_{tot}/dt=0$$\end{document} , one obtains,

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} \alpha N \,e^{-A_{tot}/(N\,A_\textrm{max})} \simeq \frac{A_{ tot}}{\tau A_*} \left[ m \left( \frac{A_{ tot}}{N} - A_0 \right) \right] . \end{aligned}$$\end{document}

We assume, for simplicity, that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_0$$\end{document} is very small, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_0 \ll A_{tot}/N$$\end{document} , while \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_\textrm{max}$$\end{document} is large, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_\textrm{max} \gg A_{tot}/N$$\end{document} . In a steady state, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$dN/dt=0$$\end{document} and the engrams do not overlap. This is because any hit in the overlapping area decreases the number of distinct centres N. Therefore, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$m=1$$\end{document} , and Eq. (16) implies that the following relationship holds true in the steady state:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} N^2 \approx \frac{A_{ tot}^2}{\alpha \tau A_* } . \end{aligned}$$\end{document}

Let us estimate the maximal number of distinct engrams. In this case, the total coverage is close to maximal, since otherwise there is a room for an additional engram in the unoccupied area, which contradicts to the assumption that the number of engrams is maximal. Therefore \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tot} \simeq A_* =2 \pi ^{d/2}/\Gamma (d/2)$$\end{document} , which finally yields

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\begin{aligned} N_\textrm{max}(d) \simeq \sqrt{ \frac{2 \pi ^{d/2}}{\alpha \tau \, \Gamma (d/2)}} . \end{aligned}$$\end{document}

The dependence \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$N_\textrm{max} (d)$$\end{document} , given by Eq. (18) is shown in Fig. 6 (Right panel). As it may be seen from the figure, the qualitative theory adequately predicts the critical dimension observed in experiments. Strictly speaking, one should use the relation, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tot}=A_*\varphi _d$$\end{document} , where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varphi _d =\varphi \left( d, f(A)\right)$$\end{document} is the “packing fraction”, which depends on the dimension and on the distribution of engrams’ areas f(A). The latter has a form that is similar to the one shown in Fig. 2, with a broad distribution of engrams’ areas—from very small values to very large ones. For the 3-d systems, it is known that the packing fraction tends to 1 (the void fraction vanishes) with increasing dispersion of spheres’ radii, see e.g.^35,36^. Therefore we use here a reasonable conjecture, that \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\varphi _d \approx 1$$\end{document} , since the distribution f(A) demonstrates a large dispersion of engrams’ areas with the abundance of small-size engrams.

Note, that the derivation of Eq. (18) for the maximal number of independent engrams is based on the kinetic equation (15) for the total areas of all engrams. Therefore, Eq. (15), and hence Eq. (18), remain valid for the case of non-uniform distribution of hits over the conceptual space and space-correlated stimuli, since it deals with \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{tot}$$\end{document} —the integral property which is not sensitive to reasonable fluctuations in spatial characteristics of engrams’ distribution. One can also consider the case of time-correlated stimuli that will also yield (18). In the SM we demonstrate that all the above cases—with space non-uniform stimuli, space and time-correlated stimuli result in the same Eq. (18), but with the renormalized hit rates \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau _*$$\end{document} .Fig. 6. Left panel: The steady state number of different engram centers (NDC), as the function of d—the dimension of the conceptual space. A critical dimension \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c$$\end{document} exists where the number of different engrams is maximal. For \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d > d_{c}$$\end{document} the number of different centers decreases with dimension tending to one as \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d \rightarrow \infty$$\end{document} . Middle panel: The critical dimension \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c$$\end{document} as a function of the forgetting rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} . \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c$$\end{document} decreases with increasing \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and saturates at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c = 7$$\end{document} . Right panel: The maximal number of distinct engrams as a function of the dimension of the ambient space, as prescribed by Eq. (18).

Kinetics of engrams with inhibition

Let us consider now somewhat more detailed model for evolution of an engram group, when a conceptual hit acts on intersecting engrams in the conceptual space. The engrams differ by the distance of their centers from the hit point in the conceptual space. Then we consider two models, associated with the (i) closest distance between the engram center and hit point and (ii) with the largest distance. For both models the centers of all hit engrams move to the hit point. However, for model, (i), only the engram with the shortest distance shrinks, according to the Poisson distribution, while for the second model, (ii), only the engram with the longest distance shrinks. We call this model engrams with inhibition.

Similarly to the basic model we performed MC simulation for the kinetic model with inhibition, see Supplementary Materials for detail. We observe the same qualitative behaviour as found for the basic model. At small time t, number of engrams with different centers increases with dimension, but with growing time it becomes non-monotonic function of d, as it is illustrated in Fig. 7. Furthermore, for small focusing constant \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_0$$\end{document} the basic model and model with inhibition predict different steady state number of engrams, although for large \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_0$$\end{document} steady states for basic and two models with inhibition coincide; the behavior of the average overlapping area with time is also similar to this of the basic model, see Supplementary Materials.Fig. 7. The number of engrams with different centers (NDC) over dimension d for different time, for two models with inhibition. Left panel—the model (i), when only engram with the minimum distance of its center from the hit point shrinks. Right panel—the model (ii), when only engram with the largest distance of its center from the hit point shrinks (see the text for more detail). The maximum area \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{max}$$\end{document} is \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_{max} = (\pi ^{d/2}/2\Gamma (d/2))$$\end{document} . Matlab R2022b was utilized to generate this figure.

Methods

In our numerical experiments, we used Monte Carlo simulations to produce trajectories of relevant variables characterising memory engrams. Algorithm 1 presents numerical workflow for the basic model. Algorithm 2 describes the process we used to simulate the kinetic model with inhibition and Algorithm 3 describes the Monte Carlo simulation steps for the one-dimensional process of learning from scratch.

Algorithm 1Monte Carlo Simulation for d-dimensional model

Algorithm 2Monte Carlo Simulation for d-dimensional model with inhibition

Algorithm 3Monte Carlo Simulation for the one-dimensional model for Learning from scratch

Discussion and conclusion

This work introduces kinetic models of memory engrams. Engrams in our models are represented by evolving geometrical objects in d-dimensional conceptual space. These objects are not rigid as several engrams can intersect, partially occupying the same region in the conceptual space, or can even form nested structures, when one engram resides inside a bigger one. The engrams respond to particular “impressions”—stimuli from the external world, which hit points of the conceptual space. If a stimulus hits an engram, the latter responds, otherwise the hit does not change the conceptual space. The stimuli hitting the engrams change their size and location. This mimics the learning process in terms of engrams. The action of a stimulus (hit) onto an engram has two consequences: (i) it confirms the corresponding concept and rectifies it, and (ii) it makes the concept more focused. The first action (i) is modelled by the shift of the geometric center on an engram to the point of the hit. The second action (ii) is modelled by the shrinkage of the engram to smaller objects in the conceptual space. In between stimuli engram permanently and steadily enlarge. This process mimics the forgetting of a concept—the engram becomes less and less focused. Hence in our model we introduce a set of kinetic coefficients, characterising the forgetting rate and the focusing rate of learning.

Based on this simple geometric model of memory and learning by engrams, we derived a kinetic equation for the characteristic function of engrams which describes the engram size distribution in the conceptual space. We start from a toy-model of one-dimensional conceptual space and find an analytical steady-state solution of the engram kinetic equation. The analytical result is in an excellent agreement with the respective numerical solution of the engram equation. Then we consider d-dimensional conceptual space and solve the engram kinetic equation by means of Monte Carlo approach. We analysed the time evolution of the engram characteristic function and observe that it tends, in the course of time, to a steady state distribution. Both evolution of the function and its steady state sensitively depend on the kinetic coefficients of the model—on the forgetting rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and focusing constant \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$A_0$$\end{document} . We analyze in detail such an important characteristic of the engram ensemble, as the number of different centers of engrams in conceptual space. We observe that this quantity is not a monotone function of dimension of the ambient space. Instead, it peaks at an extremal dimension \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c$$\end{document} . The value of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c$$\end{document} decreases with the forgetting rate \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} and saturates at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c=7$$\end{document} for larger values of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\alpha$$\end{document} .

To understand the phenomenon of critical dimension, we developed a mean-field theory of the evolution of engrams. The new theory offers an explanation for the existence of critical dimension. Moreover, it gave a correct quantitative prediction of the value of the critical dimension ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d_c=7$$\end{document} ). Through numerical simulations, we found that for small forgetting rate and strong focusing, the number of engrams with different centers is large, while the size of engrams in the steady state is relatively small. This corresponds to the sharpening of concepts—all engrams become very focused. At the same time, a significant part of the conceptual space remains uncovered. This means that a large fraction of stimuli remains unnoticed by the system, which corresponds to a relatively poor learning. At the opposite end of the spectrum, when forgetting rate is large and focusing is weak, the engrams are relatively large and the uncovered area of the conceptual space is relatively small. This corresponds to much more effective learning, when most of the stimuli cause changes in the conceptual space. However, the learned concepts are not sharp and many engrams overlap. Hence in the studied engram model of learning we observe an inherent trade-off between the receptivity to new stimuli (“readiness for learning”) and sharp selectivity of learning. This is reminiscent of classical bias-variance trade-off in statistical learning theory^37^.

In addition to revealing and justifying the existence of critical dimension in the space of concepts, the proposed kinetic model of the dynamics of memory engrams may offer novel interpretation of empirical data, both existing and new. For example, one can empirically examine and explore the mechanisms of engrams’ merging and fragmentation. Suppose two different yet sufficiently close (in conceptual space) engrams are formed in a subject. Then, at later time, the same subject receives a new stimulus (an impression) which could be associated with either of the two engrams. The research question would be to investigate the probability of merging these two concepts into a single one as a function of both timing between the events and intensity of stimulation. Depending on the outcomes (preferential sharpening of one engram over the other, merging, or dissociation of engrams) we would then be able to verify a particular regime of engrams’ response and interaction. One can also imagine a hypothetical experiment when a subject is exposed to an external stimulus (a new experience which could be viewed as a precursor of a memory concept) at a time \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t_0$$\end{document} . If this stimulus is sufficiently strong/important then we may assume that it will produce a sharpened reflection in the subject’s mind corresponding to a concept. After some time, at \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t>t_0$$\end{document} in the same experiment, we expose the subject to another stimulus. After some time, we can then ask the subject report if the second stimulus was detected or not. The minimal value of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$t-t_0$$\end{document} corresponding to the pairs of stimuli which the subject was able to detect could server as an estimate of the value of \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\tau$$\end{document} in our model.

In addition to basic kinetic model of evolving engrams corresponding to a system with pre-existing engrams (confirming learning), we also studied two related models: learning from scratch and kinetic engrams with inhibition. In the former model, new engrams emerge when an impression hits the same area of the conceptual space several times in a row. In the latter model, we assume that when an impression hits overlapping engrams, only one engram whose center is the closest to the location of the hit experiences focusing shrinkage. We have found that all the studied extensions of the basic model have qualitatively similar behavior as the basic kinetic engram model. One of the intriguing consequences of the model is the apparent existence of the optimal number of senses in evolving neural and neuromorphic systems. Indeed, suppose that the number of features associated with stimuli has a one-to-one correspondence with the number of human or artificial agent’s senses. Then the dimension of engrams in the conceptual space coincides with the number of senses. In this case, the largest capacity of the conceptual space, that is, the most rich perception of the external world, would be attained when the number of senses is equal to 7 - the critical dimension where the number of survived/retained different concepts is maximal.

To conclude, we believe that out kinetic engram model, based on the geometric representation of engrams in conceptual d-dimensional space will allow to understand qualitatively basic properties of learning and memory.

Supplementary Information

Supplementary Information.