On the validity of memristor modeling in the neural network literature

TL;DR

This paper critically examines existing models labeled as memristive in neural network research, revealing that many do not accurately represent true memristive behavior, thus questioning their validity.

Contribution

It clarifies the distinction between genuine memristive models and non-memristive alternatives used in neural network literature, highlighting the need for proper modeling.

Findings

Many models are non-memristive, describing resistors or bi-state systems.

A significant portion of literature uses models unrelated to true memristors.

Questionable relevance of some published results to actual memristive neural networks.

Abstract

An analysis of the literature shows that there are two types of non-memristive models that have been widely used in the modeling of so-called "memristive" neural networks. Here, we demonstrate that such models have nothing in common with the concept of memristive elements: they describe either non-linear resistors or certain bi-state systems, which all are devices without memory. Therefore, the results presented in a significant number of publications are at least questionable, if not completely irrelevant to the actual field of memristive neural networks.

Equations

$$R_{M}^{(1)}(\dot{V}_{M}(t))=\left\{\begin{array}[]{ccc}R_{on},&\dot{V}_{M}(t)>0&\\ R_{off},&\dot{V}_{M}(t)<0&,\\ \textnormal{unchanged},&\dot{V}_{M}(t)=0&\end{array}\right.$$

$$R_{M,ij}^{(2)}(V_{j})=\left\{\begin{array}[]{ccc}\hat{R}_{ij},&|V_{j}|>T_{i},\\ \check{R}_{ij},&|V_{j}|<T_{i},\\ \end{array}\right.$$

$$I$$

$$\dot{\mathbf{x}}$$

Full text

This Letter refers to a number of publications on “memristive” neural networks (MNNs) published during the last decade [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60] (note that this list may be incomplete, as there may be other publications that slipped through our search). We put the word “memristive” in quotes, because as we will show in the present paper, the referenced published papers refer to models that have nothing to do with resistive memories (memristive elements).

In fact, in Refs. [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60], two types of non-memristive models were used in the modeling/simulation of MNNs. Our main statement in this work is that the devices considered in these publications have no memory of past dynamics, and as such they cannot represent memristive elements. Consequently, the results obtained with these models have no relevance to the field of actual memristive neural networks [61].

To simplify the presentation, we will refer to the aforementioned models as “type 1” and “type 2” models. The type 1 model [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21] claims to approximate a “memristive element” by an expression of the type

[TABLE]

where $R_{M}^{(1)}$ is supposed to be the memristance (memory resistance), $V_{M}(t)$ is the voltage across the device, $R_{on}$ and $R_{off}$ are the low- and high-resistance states of the device, respectively, and the dot denotes the time derivative. To the best of our knowledge, the first use of Eq. (1) was proposed in Ref. [12].

In the type 2 model [22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55], the memristance in a MNN is represented by an expression of the form

[TABLE]

where $T_{i}$ are thresholds, $\hat{R}_{ij}$ and $\check{R}_{ij}$ are constants, and $V_{j}$ is the voltage at a node $j$ of the network.

Based on a literature search, the model represented by Eq. (2) was pioneered by the authors of Ref. [37]. Moreover, there is a sub-set of publications [56, 57, 58, 59, 60] where both type 1 and type 2 models are mentioned. While Eqs. (1) and (2) look different, they have a feature in common: the devices that they describe are not memristive elements.

To proceed, let us first recall the definition of actual memristive elements [63]. These are two-terminal resistive devices with memory defined (in the voltage-controlled case [63]) by

[TABLE]

where $I$ and $V_{M}$ are the current through and voltage across the device, respectively, $R_{M}\left(\bm{x},V_{M}\right)$ is the memristance (memory resistance), $\bm{x}$ is an $n$-component vector of internal state variables, and $\bm{f}\left(\bm{x},V_{M}\right)$ is a vector function.

The memory feature of memristive elements is related to their internal state that evolves according to Eq. (4) and is manifested in the device response (notice that $R_{M}$ is a function of $\bm{x}$). When subjected to time-dependent input, memristive elements typically exhibit pinched hysteresis loops. Importantly, due to the presence of memory, these loops must be strongly dependent on the input frequency (and voltage amplitude) [63, 64]. Note that this is physically necessary for any system with memory [65]. For instance, for high-frequency input signals the hysteresis loop closes, as there is not enough time for the internal state variables to follow the fast-varying input.

Now, a brief comparison of Eqs. (1) and (2) with Eqs. (3) and (4) is sufficient to establish the fact that the devices described by the type 1 and type 2 models are not memristive. While the actual memristive elements are characterized by a memory (time non-locality) of signals applied in the past, the response of type 1 and type 2 devices is effectively history-independent. This feature is readily evident in the case of type 2 model that simply describes a non-linear resistor, whose resistance is fully determined by the instantaneous voltage (which, in some publications [37], is not even the voltage across the device).

In the case of type 1 models, the instantaneous response is determined by the sign of the time-derivative of the voltage. Even though the time derivative implies the dependence on the voltage at an infinitesimally close preceding moment of time, this alone is not sufficient for the device to be classified as a memristive element. We emphasize that not only does the time derivative of the voltage not enter Eqs. (3) and (4), but also it is difficult to imagine an actual physical device with such a voltage differentiation capability (definitely the physical memristive elements behave differently [66]).

Finally, consider the last line in Eq. (1), which is the condition that the response of type 1 devices is unchanged when $\dot{V}_{M}(t)=0$. Such an isolated point condition is irrelevant since it is singular.

To further emphasize the distinction between the type 1 and type 2 devices with an actual memristive model, Fig. 1 compares their response under the condition of periodic bias. Here, the memristive device is exemplified by a threshold-type model [62, 67] that mimics the most common bipolar memristive elements [66], while the response of the type 1 and 2 devices is plotted based on Eqs. (1) and (2), respectively.

First of all, consider the application of a simple sinusoidal voltage. This is shown in Fig. 1(a). The response of the type 1 device seems deceptively similar to that of an actual memristive element, but close inspection shows that such a similarity is superficial. Indeed, unlike the actual memristive element, the type 1 device exhibits frequency-independent pinched hysteresis loops in the voltage-current plane (shown in the top left inset in Fig. 2) and its switching occurs always at voltage extrema but not at the threshold voltages defined by the physical processes responsible for memory as in actual memristive elements. Frequency-independence of the I-V curve is also evident for the type 2 device as shown in the bottom right inset in Fig. 2. In addition, the non-hysteretic character of these curves indicates the absence of memory in the type 2 model.

Next, consider the response to more complex waveforms. Fig. 1(b) shows that small higher-frequency oscillations added to the main sinusoidal waveform change drastically the response of the type 1 device. Now its resistance switches at the frequency of small-amplitude signal, and has nothing in common with the behavior of an actual memristive element (whose resistance has not changed significantly compared to Fig. 1(a)). This demonstrates that the type 1 devices are highly sensitive to small amplitude variations as opposed to the actual memristive element. In Fig. 1(a) and (b), the resistance dynamics for the type 2 model involves a frequency doubling. According to the discussion above, the absence of memory in this model is evident.

To conclude, in this Letter we have shown that two types of “memristive” models widely used in the literature to model/simulate memristive neural networks are, in fact, not memristive. During the past decade, multiple studies based on these models have been reported in leading specialized journals, such as Neurocomputing, Neural Networks, etc. There are serious reasons to doubt the validity of these papers as the models adopted by their authors do not qualify as memristive, and as such have nothing to do with actual memristive neural networks.