A Pulse-Gated, Predictive Neural Circuit

TL;DR

This paper proposes a neural circuit model that uses pulse gating and Hebbian plasticity to enable flexible information routing and decision-making, resembling biological neural systems.

Contribution

It introduces a novel pulse-gated, predictive neural circuit framework that integrates Hebbian learning for adaptive information processing.

Findings

Demonstrates pulse-controlled information processing in neural circuits

Shows how Hebbian plasticity enhances circuit adaptability

Resembles biological neural structures with layered and oscillatory features

Abstract

Recent evidence suggests that neural information is encoded in packets and may be flexibly routed from region to region. We have hypothesized that neural circuits are split into sub-circuits where one sub-circuit controls information propagation via pulse gating and a second sub-circuit processes graded information under the control of the first sub-circuit. Using an explicit pulse-gating mechanism, we have been able to show how information may be processed by such pulse-controlled circuits and also how, by allowing the information processing circuit to interact with the gating circuit, decisions can be made. Here, we demonstrate how Hebbian plasticity may be used to supplement our pulse-gated information processing framework by implementing a machine learning algorithm. The resulting neural circuit has a number of structures that are similar to biological neural systems, including a layered structure and information propagation driven by oscillatory gating with a complex frequency spectrum.

Equations

$$x_t=\sum _{i=1}^n{a}_i{x}_{t-i}+{ϵ}_t.$$

$$⟨x_t{x}_t⟩$$

$$⟨x_t{x}_{t-1}⟩$$

$$⟨x_t{x}_{t-n}⟩$$

$$\mathbf{\sigma }=\Sigma \mathbf{a},$$

$$\Sigma=\left[\begin{array}[]{cccc}\sigma_{0}&\sigma_{1}&\dots&\sigma_{n-1}\\ \sigma_{1}&\sigma_{0}&\ddots&\\ \vdots&\ddots&\ddots&\\ \sigma_{n-1}&&&\sigma_{0}\end{array}\right]\;,$$

$$\tau \frac{d{m}_1}{dt}=-m_1+{\left[x-m_0+p(t)-g_0\right]}^{+}$$

$$\tau \frac{d{m}_2}{dt}=-m_2+{\left[m_0-x+p(t)-g_0\right]}^{+},$$

$$\tau \frac{d\mathbf{a}}{dt}$$

$${\mathbf{a}}_{n+1}={\mathbf{a}}_n+\frac{\Delta t}{\tau }\Sigma \left(\mathbf{\sigma }-\Sigma {\mathbf{a}}_n\right),$$

$$\Xi=\left[\begin{array}[]{cccccccc}1&-1&0&0&0&0&\dots&\dots\\ 0&0&1&-1&0&0&\dots&\dots\\ 0&0&0&0&1&-1&\dots&\dots\end{array}\right]\;,$$

$$\Gamma=\left[\begin{array}[]{cccccccc}\sigma_{0}^{++}&\sigma_{0}^{+-}&\sigma_{1}^{++}&\sigma_{1}^{+-}&\dots&\dots\\ \sigma_{0}^{-+}&\sigma_{0}^{--}&\sigma_{1}^{-+}&\sigma_{1}^{--}&\dots&\dots\\ \sigma_{1}^{++}&\sigma_{1}^{+-}&\sigma_{0}^{++}&\sigma_{0}^{+-}&\dots&\dots\\ \sigma_{1}^{-+}&\sigma_{1}^{--}&\sigma_{0}^{-+}&\sigma_{0}^{--}&\dots&\dots\\ \vdots&\vdots&\vdots&\vdots&\ddots&\ddots\\ \vdots&\vdots&\vdots&\vdots&\ddots&\ddots\end{array}\right]$$

$$\tau \frac{d\mathbf{p}}{dt}=\Gamma \left(\mathbf{\gamma }-\Gamma \mathbf{p}\right)$$

$$\tau \frac{d\mathbf{a}}{dt}=\tau \Xi \frac{d\mathbf{p}}{dt}$$

$${\mathbf{p}}_{n+1}^{\prime }={\mathbf{p}}_n^{\prime }+\frac{\Delta t}{\tau }{Q}_1{\Gamma }^{\prime }\left({\mathbf{\gamma }}^{\prime }-Q_1{\Gamma }^{\prime }{\mathbf{p}}_n^{\prime }\right),$$

$$Q_1$$

$$Q_2$$

$${\mathbf{p}}^{\prime }=(a_1^{+},a_1^{-},\dots ,a_n^{+},a_n^{-},0,0).$$

$${\tau }_s\frac{d{s}_j}{dt}=-(s_j-x(t)y(t)),$$

$$G(t)=\left\{\begin{array}[]{cc}\frac{t}{\tau}e^{-t/\tau},&0<t<T\\ \frac{T}{\tau}e^{-t/\tau},&T<t<\infty\end{array}\right.\;.$$

$$s_j(t)\approx {\hat{\sigma }}_j,$$

$$⟨x_t^{+}⟩=\sum _{i=1}^n(⟨a_i^{+}{x}_{t-i}^{+}⟩+⟨a_i^{-}{x}_{t-i}^{-}⟩)$$

$$⟨x_t^{-}⟩=\sum _{i=1}^n(⟨a_i^{+}{x}_{t-i}^{-}⟩+⟨a_i^{-}{x}_{t-i}^{+}⟩)$$

Full text

I Introduction

A considerable experimental literature indicates that 1) oscillations of various frequencies are important for the modulation of interactions in neural systems [1, 2, 3], 2) each individual pulse that makes up an oscillation may be implicated in the parallel transfer of a packet of information [4, 5, 6, 7], and 3) neural systems exist for the control of information propagation [8]. We have begun to build a neural information processing framework based on the pulse-gated propagation of graded (amplitude dependent) information [9, 10]. A key question that arises in understanding information processing in the brain is: How may neural plasticity be used to form computational circuits?

In this paper, we explore this question by creating a predictive neural circuit based on the pulse-gated control of firing rate information. We outline a set of sub-circuits responsible for sub-computations needed to estimate the process coefficients of an autoregressive ($AR$) learning circuit. We then combine the sub-circuits to demonstrate that a neural system can use Hebbian learning in concert with pulse-gating to predict an $AR$ process.

II Methods

II-A Autoregressive Processes

AR and moving-average (MA) processes are the two principle linear models that are used to make statistical predictions [11, 12]. AR models are filters of length $n$ on a time series and we will consider these here. The object of this work is to implement an algorithm for predicting an AR process in a substrate of neurons. Below, we denote the filter by $\mathbf{a}$, the individual process covariances by $\sigma_{i}$, and the process variance-covariance matrix by $\Sigma$.

An AR($n$) process for a zero-mean random variable, $x_{t}$, is defined

[TABLE]

To find the filter, we need to solve the Yule-Walker equations, which arise from the covariance structure of the process as follows:

[TABLE]

where $\langle\rangle$ denotes an expectation value over $t$. Defining $\sigma_{i}=\langle x_{t-i}x_{t}\rangle=\langle x_{t}x_{t-i}\rangle$, from the second through $n$’th equations above, we have

[TABLE]

where $\bm{\sigma}=(\sigma_{1},\dots,\sigma_{n})$,

[TABLE]

and $\mathbf{a}=(a_{1},\dots,a_{n})$.

II-B Push-Me Pull-You Neuron Pairs

In our AR process, $x_{t}$, since we are considering an implementation in a neural circuit where firing rates encode information, the input to our system will be positive semi-definite. Both negative and positive values of $x_{t}-m_{0}$, where $m_{0}$ is the mean, must be able to be represented by the circuit. We therefore define neuron pairs where one neuron of the pair represents positive values (i.e. the amplitude above the mean) and the other neuron represents negative values (i.e. the absolute value of the amplitude below the mean). At any given moment, only one neuron will encode non-zero amplitude in such a pair. In order to read in and encode positive and negative values of the input in a pair of neurons, we let $m_{0}=\langle x_{t}\rangle$,

[TABLE]

and

[TABLE]

where $p(t)$ is a square pulse of duration $T$. Here, $g_{0}$ is a threshold value and we assume that $|x-m_{0}|$ is less than the amplitude of the pulse, $p(t)$, so neuron $1$ (firing rate $m_{1}$) will only fire when the input is simultaneously pulsed and above the mean and, similarly, neuron $2$ will only fire when the input is simultaneously pulsed and below the mean. We call such a pair of neurons a push-me pull-you (PMPY) pair. Additionally, PMPY amplitudes may be exactly propagated along a chain via pulse-gating, and we will also designate pairs along the chain as PMPY pairs.

II-C Recursive, Gradient Descent Solution to $\bm{\sigma}=\Sigma\mathbf{a}$

To make predictions, we need to estimate $\mathbf{a}$ for the process. To solve $\bm{\sigma}=\Sigma\mathbf{a}$, we find the (unique) zero of $\bm{\sigma}-\Sigma\mathbf{a}$ using gradient descent:

[TABLE]

Discretizing to first order in time gives

[TABLE]

and, since the eigenvalues of the symmetric matrix $\Sigma$ are positive and as long as $\Delta t/\tau$ is sufficiently small, iteration will give $\mathbf{a}_{\infty}\rightarrow\Sigma^{-1}\bm{\sigma}$ as $\tau\rightarrow\infty$.

In order to see how this solution may be computed using positive-only elements such as when performing the calculation with a set of PMPY pairs, we consider a pair of positive, semi-definite time series derived from $x_{t}$, $(x^{+}_{t},x^{-}_{t})$, where $x^{+}_{t}-x^{-}_{t}=x_{t}$. That is, the part of $x_{t}$ that is greater than the mean in a PMPY pair is contained in $x^{+}_{t}$, and the absolute value of the part of $x_{t}$ that is less than the mean is contained in $x^{-}_{t}$. The covariances associated with this time series are $\sigma^{++}_{i}\equiv\langle x^{+}_{t-i}x^{+}_{t}\rangle$, $\sigma^{+-}_{i}\equiv\langle x^{+}_{t-i}x^{-}_{t}\rangle$, $\sigma^{-+}_{i}\equiv\langle x^{-}_{t-i}x^{+}_{t}\rangle$, and $\sigma^{--}_{i}\equiv\langle x^{-}_{t-i}x^{-}_{t}\rangle$. Note that $\sigma_{i}=\sigma^{++}_{i}-\sigma^{+-}_{i}-\sigma^{-+}_{i}+\sigma^{--}_{i}$. Note also that if we define

[TABLE]

[TABLE]

$\bm{\gamma}=(\sigma_{1}^{++},\sigma_{1}^{+-},\sigma_{2}^{++},\sigma_{2}^{+-},\dots)$, and $\mathbf{p}=(a_{1}^{+},a_{1}^{-},a_{2}^{+},a_{2}^{-},\dots)$, then $\Xi\bm{\gamma}=\bm{\sigma}$, and $\Xi\Gamma\Xi^{T}=\Sigma$. Thus, if we first compute the solution to

[TABLE]

then, $\Xi\mathbf{p}=\mathbf{a}$. It may be seen directly that this gradient descent projects to the one described above since

[TABLE]

The reason this works is that multiplication of a vector, $\mathbf{v}$, by $\Xi$ returns a new vector containing the differences of each pair of elements of $\mathbf{v}$, and the matrix product $\Xi\mathbf{X}\Xi^{T}$ returns a new matrix containing the difference of the sum of diagonal elements of each $2\times 2$ block submatrix and the sum of its off diagonal elements.

Finally, it will be useful in the discussion of the neural circuit used to implement gradient descent to extend the above considerations by using a matrix $\Gamma^{\prime}$, which is defined as in (1), but based on an $(n+1)\times(n+1)$ matrix $\Sigma$. Thus, $\Gamma^{\prime}$ will contain covariances up to a lag of $n+1$. In this case, we can explicitly write the gradient descent algorithm as

[TABLE]

where $\bm{\gamma}^{\prime}\equiv Q_{2}\Gamma^{\prime}\mathbf{u}$, $\mathbf{u}\equiv(1,0,0,0,\dots)$,

[TABLE]

(i.e. $Q_{2}$ is a matrix of zeros, except for a $2$nd superdiagonal of $1$’s), and

[TABLE]

We have done this seemingly nonsensical extension because now $\gamma^{\prime}$ is generated by a vector input to the covariance matrix $\Gamma^{\prime}$, as opposed to being a fixed vector. Thus, the algorithm will work for an arbitrary AR(n) time series covariance.

II-D The Neural Circuit

Throughout our circuits, we use the pulse-gated propagation of firing rates described in [9].

II-D1 Computing Lagged Covariances

The first major structure in the circuit is set up to use Hebbian plasticity to calculate covariances, $\sigma_{i}^{++}$, $\sigma_{i}^{+-}$, etc., between successively delayed random variables in the input. To do this, the input, $x(t)$, is discretized and bound via a pulse into the circuit as current or firing rate amplitude packets, $x^{+}_{i}\equiv[x(t_{i})-m_{0}]^{+}$ and $x^{-}_{i}\equiv[m_{0}-x(t_{i})]^{+}$, in PMPY pairs. These packets are propagated in a chain of $n+1$ pairs. Additionally, two copies of each pair are made using a pulse gated copying procedure.

Hebbian plasticity is meant to mimic synaptic plasticity. It is a phenomenon in which coincident spiking activity between two neurons serves to increase the synaptic weight between the neurons. Since the probability of two spikes being coincident is proportional to the product of the firing rates, over time the synaptic strength becomes an estimate of the covariance between the firing rates of the two neurons (or in our case, neuronal populations). In our AR circuit, the populations of copied PMPY pairs interact with all-to-all connectivity (i.e. both populations in each pair in the first set of copies are connected with both populations in each pair in the second set of copies, see Fig. 1).

Synaptic weights, $\{s_{j}\}$, are set according to

[TABLE]

where $x$ and $y$ are firing rates from two neuronal populations. We assume that the input is in the form of a train of delta functions, $z(t)=\sum_{i}z_{i}\delta(t-t_{i})$, that are read in by a pulse-gated neural population. After gating into a neural population, we have $x(t)=z(t)\ast G(t)$, where $G(t)$ is the pulse envelope from the gating operation,

[TABLE]

In the limit that the Hebbian timescale is much greater than the synaptic timescale, $\tau_{s}\gg\tau$, and the gated pulses overlap only negligibly, we have

[TABLE]

where $\hat{\sigma}_{j}=4\tau/\tau_{s}\sum_{i}z_{i}z_{i-j}$ is an estimate of the covariance between $x$ and $y$ over $\tau_{s}/4\tau$ samples.

Once the synaptic strengths in the copy populations have reached an equilibrium, and using pulse gating such that the synapses are feedforward between PMPY copy $1$ and PMPY copy $2$, we have $\mathbf{c}^{2}=\Gamma\mathbf{c}^{1}$. Here, for simplicity, we have discarded the ′ and just write $\Gamma$.

II-D2 Gradient Descent

To implement the gradient descent part of the algorithm, we implement (3) using pulse gating. A method to do this is shown in Fig. 2. Here, a vector of PMPY pairs containing $\mathbf{p}_{n}$ is gated into the first copy of lagged values from the time series. The synaptic connectivity, learned using Hebbian plasticity, that contains an estimate of the covariance matrix is used (along with appropriate pulses) to perform the operation $\mathbf{c}^{2}:=Q_{1}\Gamma\mathbf{p}_{n}$. This result is then stored in a short-term memory. During this calculation, but after sufficient delay that $\mathbf{c}^{1}$ may be reused, $\mathbf{u}$ is gated into $\mathbf{c}^{1}$ and, via the matrix $\Gamma$ and another connectivity matrix that encodes $Q_{2}$, $Q_{2}\Gamma\mathbf{u}$ is computed and stored in $\mathbf{y}$. The two computation streams are then combined with appropriate inhibition to subtract $\mathbf{m}_{3}$ from $\mathbf{y}$ giving $\mathbf{z}=\gamma-Q_{1}\Gamma\mathbf{p}_{n}$. Two subsequent operations result in $\mathbf{p}_{n+1}$ being set to $\mathbf{p}_{n}+\frac{\Delta t}{\tau}Q_{1}\Gamma\left(\bm{\gamma}-Q_{1}\Gamma\mathbf{p}_{n}\right)$

II-D3 Prediction

To predict future values of $x(t)$, we need to form estimates of $x(t)$ using the AR(n) process coefficients. The computation to do this requires us to compute

[TABLE]

and

[TABLE]

using the coefficients, $\mathbf{p}=(a_{1}^{+},a_{1}^{-},\dots)$, that we have estimated with the gradient descent algorithm. One way that this may be done is depicted in Fig. 3.

Here, by initializing a set of populations to $1$ ($1$-populations), connecting them one-to-one with the elements of $\mathbf{p}_{n}$, Hebbian plasticity causes the synapses to encode the values of the elements of $\mathbf{p}$. This is a method for creating a long-term memory. Subsequently, $\mathbf{x}$ is copied to the $1$-populations, then gated to compute $\{\langle a_{i}^{+}x_{t-i}^{+}\rangle,\langle a_{i}^{-}x_{t-i}^{-}\rangle,\dots\}$. These values are then summed to compute $\langle x_{t}^{+}\rangle$. By swapping the $+$ and $-$ elements of $\mathbf{x}$, then copying to the $1$-populations, the same circuit results in $\langle x_{t}^{-}\rangle$. The output may be stored in memory or used in other sub-circuits in the usual way.

III Results

In Fig. 4, we depict results for the $AR(2)$ process, $x_{n+1}=a_{1}x_{n}+a_{2}x_{n-1}+\epsilon$, where $\epsilon$ is a zero mean Gaussian noise process, $a_{1}=3/4$ and $a_{2}=-1/2$. At the beginning of the sequence, $AR$ input to short term memory is silenced to allow operation of the gradient descent circuit. At the same time, the process coefficients, encoded in firing rate amplitudes are propagated to the coefficient learning and prediction circuit to be encoded in synaptic weights. Once the gradient descent circuit and learning and prediction circuits have updated the process coefficient estimates, $a_{1}$ and $a_{2}$, information needed for memory is propagated internally to each circuit and $AR$ input is allowed to enter the covariance learning circuit. This update sequence is then repeated to obtain progressively better estimates over a long time ($100,000$ repeats for this simulation).

In Fig. 5, we show the convergence of the algorithm to steady state estimates for both the covariance matrix elements and $AR$ process coefficients. Final $AR$ process coefficients are within $6\%$ of theoretical values. Better estimates may be obtained by increasing the learning time constant, $\tau$.

IV Conclusion

Using a pulse-gating paradigm for propagating information in a neural circuit in concert with Hebbian learning, we have described how to implement an algorithm for predicting future values of an arbitrary $AR(n)$ process. We have implemented the algorithm for an $AR(2)$ process. The algorithm consists of three sub-circuits responsible for 1) learning the covariance matrix, 2) computing the process coefficients and 3) making a prediction by learning the process coefficients, then performing the prediction.

Using only pulse-gating, Hebbian learning and standard neuronal synaptic properties, we implemented a short-term memory, a gradient descent algorithm, a long-term memory and a method for computing an inner product to make a prediction. Additionally, the structure of the algorithm defines the brain rhythms that arise during information processing. In Fig. 6, we show the spectra of the gating pulses in the pulse sequence. For this figure, we used pulses of $10$ms duration, resulting in strong gamma and theta rhythms. To our knowledge, this is the first time a neuronal model has been used to relate algorithmic structure with neuronal oscillation structure.

Acknowledgment

L.T. thanks the UC Davis Mathematics Department for its hospitality. A.T.S. would like to thank Liping Wei and the Center for Bioinformatics at the College of Life Sciences at Peking University for their hospitality. This work was supported by the Natural Science Foundation of China grant 91232715 (YXS and LT), by the Open Research Fund of the State Key Laboratory of Cognitive Neuroscience and Learning grant CNLZD1404 (YXS and LT), and by the Beijing Municipal Science and Technology Commission under contract Z151100000915070 (YXS and LT).