Memristive Devices for Computation-In-Memory

TL;DR

This paper explores the use of memristive devices to develop computation-in-memory accelerators, demonstrating significant improvements in latency, energy, and area over current architectures.

Contribution

It introduces two novel computation-in-memory accelerators utilizing memristive devices, advancing the field of emerging device-based computing architectures.

Findings

Significant reduction in latency compared to traditional architectures

Lower energy consumption achieved with memristive accelerators

Smaller area footprint of the proposed accelerators

Abstract

CMOS technology and its continuous scaling have made electronics and computers accessible and affordable for almost everyone on the globe; in addition, they have enabled the solutions of a wide range of societal problems and applications. Today, however, both the technology and the computer architectures are facing severe challenges/walls making them incapable of providing the demanded computing power with tight constraints. This motivates the need for the exploration of novel architectures based on new device technologies; not only to sustain the financial benefit of technology scaling, but also to develop solutions for extremely demanding emerging applications. This paper presents two computation-in-memory based accelerators making use of emerging memristive devices; they are Memristive Vector Processor and RRAM Automata Processor. The preliminary results of these two accelerators show significant improvement in terms of latency, energy and area as compared to today's architectures and design.

Equations

$$s[n]=\mathbf{i}\cdot {\mathbf{V}}_{\mathbf{n}}=\sum _{k=0}^{2^W}i[k]v_n[k],\forall n\in [1,N]$$

$$\mathbf{V}=\left[\begin{array}{ccc}{\mathbf{V}}_1& {\mathbf{V}}_2& {\mathbf{V}}_3\end{array}\right]=\left[\begin{array}{ccc}1& 0& 0\\ 1& 0& 1\\ 1& 1& 0\\ 0& 0& 0\end{array}\right].$$

$$f[n]=\mathbf{a}\cdot {\mathbf{R}}_{\mathbf{n}}=\sum _{i=0}^{N-1}a[i]R_n[i],\forall n\in [1,N].$$

$$a[n]=f[n]\&s[n],\forall n\in [1,N].$$

$$\mathbf{R}=\left[\begin{array}{ccc}{\mathbf{R}}_1& {\mathbf{R}}_2& {\mathbf{R}}_3\end{array}\right]=\left[\begin{array}{ccc}0& 1& 1\\ 0& 0& 1\\ 0& 0& 0\end{array}\right].$$

$$A=\mathbf{a}\cdot {\mathbf{c}}^{\top }=\sum _{n=0}^{N-1}a[n]c[n].$$

Full text

I Introduction

Today’s and new emerging applications, such as data-intensive/big-data applications (e.g., DNA sequencing) and internet-of-things (IoT), are extremely demanding with respect to computing power, energy consumption, and storage. These applications will not only strongly shape our near future, but also impact the semiconductor and computer industry. However, their requirements are difficult to fulfill with today’s CMOS based computer architectures, as they face sever challenges both at architectural and device level. Current computer architectures face three walls [1]: (1) the memory wall due to the growing gap between processor and memory speed and the the limited memory bandwidth; (2) the power wall as the practical power budget for cooling has been reached; (3) the instruction-level parallelism (ILP) wall due to the growing difficulties in extracting enough parallelism in software/code that can run on the mainstream parallel hardware today. The CMOS devices also face three walls [2]: (1) the leakage wall as the static power is becoming dominant at small technology nodes (due to volatile technology and low Vdd) and it may even be higher than the dynamic power, (2) the reliability wall as technology scaling leads to reduced device lifetime and higher failure rate; (3) the cost wall as the cost per device from a pure geometric scaling of technology point of view is plateauing. Both architecture and device walls have slowed down the performance gains of CMOS-based architectures. All these motivate the need to look for alternative architectures while considering emerging device technologies.

Many alternatives architectures are under investigations. Resistive computing [3, 4, 5] and neuromorphic computing architectures [6, 7] using memristive devices, and quantum computing using quantum dots [8] are couple of examples. Resistive computing architectures based on memristive devices are attractive, as they enable in-memory computing (reducing the memory wall) [2, 9]. In addition, the memristive devices have zero standby power [6] (helps reducing both the leakage and power wall), great scalability (reduces the cost wall), high density (reduces the cost wall), and they are CMOS compatible (reduces the cost wall).

This paper discusses two memristive device based accelerators to demonstrate how computation-in-memory architectures can realize significant improvements, due both to the architecture itself as well as to the used technology to implement them. First, a memristive based vector processor, referred to as Memristive Vector Processor (MVP), is presented; MVP can be used as an accelerator for conventional machines and shows approximately one order of magnitude improvement in performance and energy efficiency. Thereafter, a general model for hardware-based automata processing is introduced and implemented with memristive devices. This implementation is referred to as RRAM-AP; RRAM-AP’s key kernel (i.e., the vector dot product operator) outperforms the state-of-the-art SRAM-based implementation by 40% less delay and 27% less energy, at even smaller chip area.

The reminder of this paper is organized as follows. Section II describes briefly the fundamentals of memristive devices. Section III and IV present MVP and RRAM-AP, respectively. Finally, Section V concludes the paper.

II Basics of Memristive Devices

The memristive device, or memristor for short, is the fourth type of fundamental two-terminal electrical components, next to the resistor, capacitor, and inductor. It was initially predicted in 1971 by the circuit theorist Leon Chua [10]. He observed a missing element that can be described as a function of flux $\phi$ and charge $q$, as shown (with the dashed line) in Fig. 1a. In theory, a memristive device is a passive element that can be described by the current integral (charge $q$) through or voltage integral (flux $\phi$) across its two terminals; The beauty of the memristive device is its ability to memorize the history (i.e., the internal state). The essential fingerprint of memristive devices is the ‘pinched current-voltage hysteresis loop’, as illustrated in Fig. 1b. When a memristive device is floating or when the voltage v(t) across it equals zero, the current i(t) is also zero. Therefore, based on its hysteresis curve, the memristor has at least two distinctive states: a high ($R_{H}$) and low ($R_{L}$) resistive state. A memristive device switches from high (low) to low (high) state by applying a voltage $V_{\text{SET}}$ ($V_{\text{RESET}}$) with an absolute value larger than its threshold voltage $V_{\text{th}}$. Another signature of the memristive devices is that the ‘pinched hysteresis loop’ shrinks with a higher excitation frequency f as shown in Fig. 1b. Fig. 1c shows the two typical symbols used to denote memristive devices; the black square represents the positive terminal.

After a silent period for more than thirty years, a practical memristive device was fabricated and demonstrated by HP in 2008 [11]. HP built a metal-insulator-metal device using titanium oxide as an insulator and identified the memristive behaviour over its two-terminal node as described by Leon Chua; as shown in Fig. 1d. The device resistance is modulated by controlling positive charged oxygen vacancies in the insulator layer using different voltages. After the first memristive device was fabricated, several memristor devices based on different types of materials have been proposed such as spintronic, amorphous silicon, and ferroelectric memristors [6].

III Memristive Devices for Vector Processing

Memristor-based Computation-In-Memory (CIM) concept was proposed to eliminate the communication between the CPU and memory by leveraging memristors for both storage and computation in the same physical crossbar [3, 12, 13]. Here, we use the CIM to realize an accelerator we refer to as Memristive Vector Processor (MVP). The rest of this section will describe the working principle of MVP, the targeted applications and some analytical evaluation results to show the potential of such an architecture.

III-A Working principle

MVP is proposed to accelerate applications with a huge number of vector operations. It can be used as an accelerator for a conventional processor, as shown in Fig. 2a. Similarly as in conventional architectures, the processor fetches, decodes and executes a program using a memory hierarchy consisting of cache(s), DRAM, and external memory. The part of the program which is memory intensive will be offloaded to MVP. The distinct feature of MVP is its crossbar memory implementation using memristive devices, which enables not the storage of huge amount of data (due to its nano scale size), but also the processing of operations within the memory (i.e., no need for data movement).

The processing in MVP is performed based on scouting logic operations [14, 5] ; they transform memory read operations into logical operations. Normally, when a memory cell is being read, a read voltage $V_{r}$ is applied to the activated row as shown in Fig. 3a. Subsequently, a current will flow through the bit line to the input of the sense amplifier (SA) where it is compared to a reference current. Depending on the cell value (either low ($R_{L}$) or high ($R_{H}$) resistance), the output of the SA will produce either logic 1 or 0. Inspired by this read operation, scouting logic is able to implement OR, AND and XOR gates. Instead of reading a single memristor at a time, scouting logic activates two (or more) memory rows simultaneously. As a result, the input current to the sense amplifiers is determined by the equivalent input resistance of the activated rows. This resistance results in three possible values: $R_{H}$, $R_{H}$//$R_{L}$ $\approx$ $R_{L}$, or $R_{L}/2$; by changing the reference current of the SA, different gates can be realized (as shown in Fig. 3b). Therefore, using this scheme allows MVP to perform logical operations by just a small modification of the peripheral circuit of the crossbar mememory. It eliminates the necessity of temporary registers, loading latency and energy to move data from memory to registers. It also increases the parallelism of the architecture and does not impact the the endurance of the memristive devices.

III-B Potential targeted applications

With its unique capability, MVP is able to accelerate data intensive applications. These applications consist of intensive memory accesses that consume an enormous amount of energy and degrade the overall performance due to data movements through the memory hierarchy; note that loading a word from the on-chip SRAM or off-chip DRAM costs much more energy (50x and 6400x, respectively) as compared with an ALU operation [15, 16]. Therefore, eliminating data movements/ communication significantly improves the overall performance.

An example of a program that could benefit from MVP is illustrated in Fig. 2b. The program consists of multiple loops processing a dataset that is preloaded and mapped on MVP. Each time a loop is called, the processor sends a (macro)-instruction to MVP; the instruction is locally decoded and executed. The result is returned to the processor. This feature occurs in multiple applications such as database management [17], DNA sequencing [18, 19, 20], and graph processing [21].

III-C Evaluation Results

To evaluate MVP architecture, its estimated performance is compared to a multicore architecture. The models and assumptions for the multicore architecture and MVP are similar to those in [3, 9]; e.g., the multicore architecture consists of 4 cores (ALU only), two levels of caches (32 KB L1 and 256 KB L2) and 4 GB DRAM. The MVP architecture consists of one core (ALU only), two levels of caches (32 KB L1 and 256 KB L2), 2 GB DRAM, and a MVP with a 2 GB non-volatile crossbar memory with a modified read-out circuity (as explained in [14]) in order to enable computation-in-memory. Three metrics are used for the evaluation: (1) performance energy efficiency $\eta_{PE}$ (defined by MOPs/mW), (2) energy efficiency $\eta_{E}$ (defined by pJ/op), and (3) performance area efficiency $\eta_{PA}$ (defined by MOPs/mm2).

Fig. 4 shows the results of the evaluation metrics for both architectures for different L1 and L2 cache misses (up to 60%)and by assuming that 70% of the program instructions can be accelerated on MVP (%Acc=0,7); i.e., the 30% non-accelerated instructions is executed by the conventional processor and the 70% accelerated part by MVP; see Fig. 2. As MVP architecture contains a conventional part (i.e., CPU, caches, DRAM and external memory), only 10x improvement is obtained with respect to the performance-energy efficiency. MVP architecture also achieves one order of magnitude energy efficiency improvement in comparison with the multicore architecture, and has a higher performance area efficiency. Therefore, the MVP architecture has the potential of realizing significant improvements, despite the high switching latency and low endurance of memristor devices. The improvements are the result of a significant reduction of cache and DRAM accesses, and the usage of non-volatile memory. The reduction of memory accesses leads to a lower latency and lower energy consumption, while the non-volatile memory reduces the static power practically to zero.

IV Memristive Devices for Automata Processing

Automata-based processing is widely used in diverse fields, including network security [22], computational biology [23], and data mining [24]. Its hardware implementation, referred to as automata processors (APs), has significant advantages over von Neumann architectures regarding throughput and energy efficiency as they enable computation-in-memory [25, 26, 27]. Memristive devices, which are the enablers of Resistive Random-Access Memories (RRAM) and computation-in-memory, are potential candidates for implementing the APs as it will be shown in this section. We will refer to this implementation as RRAM-AP. Moreover, it will be shown that RRAM-AP outperforms the two known hardware implementations of APs, being the Micron Automata Processor [25] which is based on SDRAM, and the Cache Automation [27] which is based on SRAM; we will refer to them by SDRAM-AP and SRAM-AP, respectively, to maintain the naming consistent with RRAM-AP. Next, we will first introduce basic knowledge and notations of automata. Subsequently, we propose a generic model for automata processors. Thereafter, we present RRAM-AP implementation, and show its superiority.

IV-A Automata Basics

A Non-deterministic Finite Automata (NFA) can be represented by a 5-tuple: $(Q,\Sigma,\delta,q0,C)$. $Q$ represents a finite set of states (which are denoted with circles in the illustrative example of Fig. 5a), $\Sigma$ is a finite set of possible input symbols (that can be used to generate an input sequence), $\delta$ is the transition function describing the set of possible transitions among the states, $q0$ is one of the states from $Q$ and presents the start state, $C$ is a subset of $Q$ and contains the final states or accepting states; they are denoted with a double circle in the state diagram af Fig. 5a as shown for the final state S3.

During operation (i.e., execution of an input sequence), some states can be active; they are denoted by $P$. Initially, $P$ equals to $q0$. At each processing step, the NFA consumes one symbol $I$ from the input sequence. Based on $I$ and $\delta$, $P$ is updated. Once all symbols of the input sequence are processed, the NFA output is determined by $P$ and $C$. If $P\cap C\neq\emptyset$, then we say that the NFA accepts the input sequence; otherwise, the sequence is rejected. The acceptance of the input sequence can be represented by a Boolean value $A$.

Homogeneous automaton is a special type of NFA that is relatively easy to implement by APs [25]. It requires that a state can only be reached by transitions with the same input symbol(s). These input symbols belong to the symbol class of this state. For example, in the NFA shown in Fig. 5b, S3 can be reached by two transitions (from S1 and S2, respectively) both with the same symbol $b$; $b$ belongs to the symbol class of S3. Here, the NFA shown in Fig. 5a is a homogeneous automaton and can be therefore redrawn as depicted in Fig. 5b. Note that the input symbols are only related to the states in homogeneous automata and not the state transitions as is the case for normal NFAs; e.g., the symbol b is not on the incoming edges/transition of the state S3 (see Fig. 5a) but rather within the node representing S3 (see Fig. 5b). Any NFA can be translated into its equivalent homogeneous automaton and therefore implemented using APs [25].

IV-B Generic Automata Processor Model

Before implementing RRAM-AP, we need to understand the key operations conducted by an AP. Therefore, we next present a generic model for APs to identify these operations. This generic model is shown in Fig. 6 and consists of three major processing steps:

Input symbol processing: It decodes each symbol $I$ (presented with $W$ bits) of the input sequence by activating only one of the $2^{W}$ wordlines, and identifies all states that have an incoming transition occurring on $I$. These states and the remaining sates are presented by column vectors called State Transition Elements (STEs), and are pre-configured based on $Q$ and the corresponding symbols (symbol class). Each STE presents one state of the $N$ states of $Q$. The result of this step is mapped to a vector called Symbol Vector $\mathbf{s}$. 2. 2.

Active state processing: It generates: (1) all the possible states that can be reached from the current active states $P$ (stored in a vector called Active Vector $\mathbf{a}$) based on these states and the transition function $\delta$ (stored in the routing matrix), and stores the result in the Follow Vector $\mathbf{f}$; (2) the next active states (i.e., Active Vector) by bit-wise ANDing $\mathbf{s}$ and $\mathbf{f}$. 3. 3.

Output identification: In order to decide about the value of $A$ (i.e., whether the input sequence is accepted or not), the intersection of $\mathbf{a}$ and the Accept Vector $\mathbf{c}$ (pre-configured based on $C$) is checked. That is, if $P\cap C\neq\emptyset$, then $A=1$ (accept), otherwise $A=0$ (reject).

Next we will elaborate the above three processing steps.

IV-B1 Input symbol processing

As mentioned, the purpose of this is to calculate the Symbol Vector $\mathbf{s}$ for each input symbol. This is done based on the selected row (from the $2^{W}$ rows) and the configuration of STEs. Let’s assume that for each input symbol, an Input Vector $\mathbf{i}$ of $2^{W}$ elements is generated where only one element is high (corresponding to the selected wordline); the remaining elements are 0. In addition, assume that the configuration of STEs can be presented by a matrix $\mathbf{V}$ where each column $\mathbf{V_{n}}$ presents the STE of the state $n$. Then the $n$th element of the Symbol Vector $\mathbf{s}$ corresponding to $\mathbf{V_{n}}$ can be calculated as:

[TABLE]

In this equation, the addition and the multiplication represent the Logic OR and AND, respectively. For the example of Fig. 5b, if we assume $\Sigma=\{a,b,c,d\}$, then,

[TABLE]

This means that S1’s symbol class is $\{a,b,c\}$, S2’s is $\{b\}$, and S3’s is $\{c\}$. If we further assume that the current input symbol is $b$, then $\mathbf{i}=[0\ 1\ 0\ 0]$, and $\mathbf{s}=[1\ 0\ 1]$. This means that $b$ is in the symbol classes of S1 and S3.

IV-B2 Active states processing

This step calculates the Follow Vector $\mathbf{f}$ which presents the possible states that can be reached from the current active states stored in the Active Vector $\mathbf{a}$. The transition function is implemented by the routing matrix as shown in Fig. 6, and can be conceptually presented as a two-dimensional vector $\mathbf{R}$. Hence, the $n$th element of Follow Vector $\mathbf{f}$ can be calculated as:

[TABLE]

The interpretation of the addition and the multiplication in this equation is the same as in Equation (1). The next active states (to be also stored in the Active Vector $\mathbf{a}$) are easily calculated by using bitwise AND operation.

[TABLE]

For the example of Fig. 5b, the matrix $\mathbf{R}$ that belongs to the transit function is

[TABLE]

This means that S1 cannot be reached from all the states ($\mathbf{R_{1}}$), S2 can only be reached from S1 ($\mathbf{R_{2}}$), and S3 from both S1 and S2 ($\mathbf{R_{3}}$). For $\mathbf{a}=[1\ 0\ 0]$ (only S1 is active), $\mathbf{f}=[0\ 1\ 1]$ according to Equation (2). This means S2 and S3 are reachable states from the active states. If we assume the next input symbol is $b$, which leads to $\mathbf{s}=[1\ 0\ 1]$ as discussed above, then the new active vector $\mathbf{a}=[0\ 0\ 1]$ according to Equation (3). This means that S3 becomes the next active state.

IV-B3 Output identification

The output value $A$ of NFA is easily calculated using the Active Vector $\mathbf{a}$ and the Accept Vector $\mathbf{c}$. The former stores the active states generated by the input sequence while the later stores the defined accepting states of NFA.

[TABLE]

$A=1$ means that the input symbol sequence is accepted by the NFA; otherwise, the string is rejected. For the example of Fig. 5b, $\mathbf{c}=[0\ 0\ 1]$. This means only S3 is an accepting state. If we assume the same example as above ($\mathbf{a}=[0\ 0\ 1]$), then $A=1$.

IV-C RRAM-AP Implementation

The automata processing model described above contains only two types of logic operations, which are vector dot product (Equation 1, 2, and 4) and vector bit-wise AND (Equation 3). In practice, we cannot implement the complete routing matrix of Equation 2, as it requires too much resource. SDRAM-AP and SRAM-AP both use hierarchical routers to implement the routing matrix. Their implementations do not support all NFA transitions; nevertheless, there is enough flexibility to route all possible transitions of typical applications [25, 27]. While SDRAM-AP does not reveal many implementation details, SRAM-AP uses a two-level structure that consists of global and local switches [27]. These global and local switches also conduct vector dot product operations.

For our implementation, we adopt SRAM-AP’s for the routing matrix, use the hardware structure shown in Fig. 7a for STEs, and the one in Fig. 7b both for global and local switches. The black and white boxes represent different configuration bits. Each column generates the vector dot product of the input vector and the configuration bits of this column.

An NFA is configured to RRAM-AP by programming RRAM devices to either low or high resistance. We use one transistor and one RRAM device (1T1R) to implement a configurable bit as shown in Fig. 8b. During the configuration, the word line WL selects the row to be programmed, and the programming voltage is applied to the bit line BL as shown in Fig. 8a. The programming voltage can be either SET or RESET voltage. Logic 1 corresponds to the memristor’s low resistance, and logic 0 to high resistance. The bit line is pre-charged before evaluation, and the word lines are selected, e.g., by the input symbols. Note that for the routing matrix, multiple word lines can be activated in parallel. The vector dot product is calculated when all the word lines are set; if all the corresponding selected cells contain a high resistance (i.e., logic 0), then the pre-charged bit line remains high, and the sense amplifier (SA) will read a logic 0 (inverted output). Similarly, if at least one of the cells contains a low resistance (i.e., logic 1), then BL will be discharged. The SA’s output will subsequently be a logic 1.

The characteristics of memristors provide opportunities for RRAM-AP to outperform previous designs. For example, SRAM-AP uses eight transistors to implement the configurable bit as shown in Fig. 8c [27], whose area is much larger than the 1T1R structure. In addition, the SRAM cells also suffers from leakage power. As memristors are non-volatile devices, RRAM-AP can resume the last configured NFA after shut down and reboot without reprogramming it. On the other hand, RRAM-AP also inherits some drawbacks, such as the longer and power-hungry programming phase, and lower endurance, in comparison with SDRAM and SRAM.

IV-D Preliminary Results

The APs can be built by using only vector dot product and bit-wise AND operators. Except for the vector dot product operator, we assume that the remaining part of RRAM-AP is implemented in a similar way as SRAM-AP (incl. bit-wise AND, wiring, and sense amplifiers). Hence, we compare only the dot product operator. Note that SRAM-AP outperforms SDRAM-AP regarding the throughput and energy consumption; therefore, we limit our comparison to SRAM-AP.

The simulated circuit consists of a single vector dot product operator with a length of 256 as shown in Fig. 9a. We use $32\text{\,}\mathrm{nm}$ PTM model for CMOS transistors and ASU model [28] for RRAM. We configure RRAM’s parameters based on a two-state device, similarly as presented in [29], e.g., the RRAM’s high and low resistances are approximately $100\text{\,}\mathrm{M\SIUnitSymbolOhm}$ and $1\text{\,}\mathrm{k\SIUnitSymbolOhm}$ respectively; the SET and RESET threshold voltages are $1.3\text{\,}\mathrm{V}$ and $0.5\text{\,}\mathrm{V}$. To simulate the slowest discharge process, only the first cell is configured to logic 1 (indicated by the black box), and the remaining 255 cells are configured to be 0 (indicated by white boxes). The bit line BL is pre-chared to $0.4\text{\,}\mathrm{V}$ (lower than RRAM’s threshold voltages). When BL is discharged to $0.1\text{\,}\mathrm{V}$, the sense amplifier (not included in the circuit) will read a 1. The reference voltage of the SA is set to $0.25\text{\,}\mathrm{V}$.

The HSPICE simulation results are shown in Fig. 9b. The word line WL is enabled at $1\text{\,}\mathrm{ns}$, and then BL starts discharging. BL’s voltages in SRAM and RRAM-based designs are illustrated with solid blue line and dashed red line, respectively. The discharge time through RRAM ($104\text{\,}\mathrm{ps}$) is 35% less than the SRAM-based implementation ($161\text{\,}\mathrm{ps}$). This is mainly because transistors have relatively large intrinsic capacitance. During bit-line discharge, the RRAM cell of Fig. 8b has only one transistor in its path while the SRAM-based design has two (See Fig. 8c). The energy consumed during the charge and discharge processes is $2.09\text{\,}\mathrm{fJ}$ for the RRAM-based design and $5.16\text{\,}\mathrm{fJ}$ for the SRAM-based design. The former is 59% less than the latter. Considering that the remainder part of RRAM-AP is implemented in a similar way as SRAM-AP, RRAM-AP outperforms SRAM-AP at the chip level regarding latency, energy, and area.

V Conclusion

In this work, we have discussed two potential applications of memristive devices and computation-in-memory, i.e., Memristive Vector Processor and RRAM Automata Processor. Memristors’ unique properties provide us an important opportunity to improve conventional designs at both architectural and device level. However, the drawbacks of memristor technology, such as the impact of endurance, require further research.

Acknowledgment

This work was supported by the European Union’s Horizon 2020 Research and Innovation Program through the project MNEMOSENE (Grant 780215).