Catwalk: Unary Top-K for Efficient Ramp-No-Leak Neuron Design for Temporal Neural Networks

TL;DR

This paper introduces Catwalk, a novel neuron implementation for temporal neural networks that leverages spike sparsity via unary top-k to improve hardware efficiency, reducing area and power consumption.

Contribution

It proposes a new neuron design using unary top-k to efficiently handle sparse spike volleys, enhancing CMOS implementation of RNL neurons.

Findings

Catwalk reduces area by 1.39x compared to existing neurons.

Catwalk reduces power consumption by 1.86x.

The approach exploits spike sparsity for hardware efficiency.

Abstract

Temporal neural networks (TNNs) are neuromorphic neural networks that utilize bit-serial temporal coding. TNNs are composed of columns, which in turn employ neurons as their building blocks. Each neuron processes volleys of input spikes, modulated by associated synaptic weights, on its dendritic inputs. Recently proposed neuron implementation in CMOS employs a Spike Response Model (SRM) with a ramp-no-leak (RNL) response function and assumes all the inputs can carry spikes. However, in actual spike volleys, only a small subset of the dendritic inputs actually carry spikes in each compute cycle. This form of sparsity can be exploited to achieve better hardware efficiency. In this paper, we propose a Catwalk neuron implementation by relocating spikes in a spike volley as a sorted subset cluster via unary top-k. Such relocation can significantly reduce the cost of the subsequent parallel counter (PC) for accumulating the response functions from the spiking inputs. This can lead to improvements on area and power efficiency in RNL neuron implementation. Place-and-route results show Catwalk is 1.39x and 1.86x better in area and power, respectively, as compared to existing SRM0-RNL neurons.

Tables

Table 1. TABLE I: Place-and-route results of different neurons in 45 nm CMOS. Neuron configurations are same as in Fig.  9 .

Neuron design	Power ($\mu W$)			Area
	Leakage	Dynamic	Total	($\mu m^2$)
$n=16,k=2$
PC conventional	5.11	94.65	99.76	245.25
PC compact [7]	4.84	96.95	101.80	239.13
Sorting PC	4.28	70.11	74.39	197.64
Top-k PC (Catwalk)	4.22	69.40	73.62	194.98
$n=32,k=2$
PC conventional	6.73	138.08	144.81	338.62
PC compact [7]	6.59	147.57	154.16	333.56
Sorting PC	5.73	88.24	93.97	256.42
Top-k PC (Catwalk)	5.66	86.79	92.45	252.97
$n=64,k=2$
PC conventional	9.39	210.79	220.19	500.88
PC compact [7]	9.29	236.20	245.50	495.03
Sorting PC	8.12	129.59	137.71	364.15
Top-k PC (Catwalk)	7.85	124.21	132.06	355.38

Equations

$$\rho (w,t)$$

$$=t+1$$

$$=w$$

Full text

I Introduction

Neuromorphic computing, particularly represented by spiking neural networks (SNNs) [15], has emerged as an attractive alternative to conventional compute-intensive deep neural networks (DNNs) due to its brain-inspired approach for computational efficiency. Unlike DNNs, which transmit continuous-valued signals, neurons in SNNs communicate information through discrete spike or pulse events. Central to these architectures is the neuron model, which defines how neurons generate and respond to spikes. Among various neuron models, the spike response model (SRM) has been widely utilized, especially in its simplified variant, the SRM0 model, characterized by a fixed spike generation threshold. The SRM0 neuron can employ several types of response functions, such as biexponential [4], piecewise linear [6], step-no-leak [12, 3, 5], and, notably, the ramp-no-leak (RNL) function [13, 12]. The RNL function has recently gained significant interest due to its practical advantages in temporal neural network (TNN) applications [7, 13, 12, 1, 17, 14, 8]. TNNs, a special class of SNNs, are capable of continuous [13] online learning and unsupervised clustering [1, 12] by employing precise spike timing and biologically-plausible spike-timing dependent plasticity (STDP) local learning rule. This contrasts with the compute intensive global backpropagation methods predominantly used in traditional DNNs.

Existing TNNs adopt an SRM0 neuron model with an RNL response function [13, 12]. For brevity, we henceforth call this neuron model an SRM0-RNL neuron in this paper. However, current SRM0-RNL neuron implementations [7, 17, 8] employ worst-case scenarios for temporal signal processing and provides increased hardware resources for maximal spike density scenarios, which rarely occurs due to the inherent sparsity, i.e., just 0.1%-10% of total count of neurons are observed to fire biologically [11, 20, 10].

To address these inefficiencies, this paper introduces Catwalk, a novel neuron architecture employing unary top-k to optimize spike aggregation at neuron inputs. Catwalk leverages the inherent sparsity in temporal-coded spikes to cluster active spikes efficiently, enabling the replacement of conventional, fully provisioned parallel counter (PC), for accumulation of input response functions, with significantly more compact and efficient counterparts. Specifically, our contributions are:

• •

We propose Catwalk, a novel technique that relocates the top-k temporal spikes to optimize spike aggregation in neuron inputs for better hardware efficiency.

• •

A detailed evaluation against baseline designs, including SRM0-RNL neurons that employ conventional PCs and those that integrate unary sorting, demonstrates the clear advantages of our proposed Catwalk neuron.

• •

Place-and-route evaluation at 45nm CMOS shows area and power improvements of $1.39\times$ and $1.86\times$, respectively, relative to existing SRM0-RNL neurons.

This paper is organized as follows. Section II and Section III review the background and motivate this work. Then Section IV describes the proposed neuron design. The following Section V and Section VI evaluate the implementation. Finally, Section VII concludes this work.

II Background

II-A SRM0-RNL Neuron

SRM0-RNL neurons closely follow the formulation of biological neurons (Fig. 1), and existing TNNs encode information through precise spike timings rather than spike rates [13, 12]. The mechanism of this neuron model is explained further in Fig. 2a from prior works [13, 12], and is contrasted with our Catwalk neuron model, depicted in Fig. 2b. Each post-synaptic neuron is fed by incoming temporal-coded spikes from all preceding neurons (pre-synaptic neurons), where the synapse responses are accumulated. The axon fires a spike if the accumulated membrane potential exceeds a certain threshold. In the existing SRM0-RNL implementations, all incoming responses are summed during potential accumulation [7]. The RNL response function is provided in Equation 1, where $w$ is the weight value and $t$ refers to time. Therefore, the RNL response function essentially creates a pulse of width $w$ over time, to be accumulated later.

[TABLE]

II-B Unary Sorting

Unary data representations, either rate-coded or temporal-coded, have been used to perform various operations, including multiplication, addition [18], division, square root [19], min, max [16]. Among these, bitonic sorting on temporal-coded unary data (e.g. temporal spikes in SRM0-RNL neurons) can be implemented via simple AND and OR gates [9]. An example of unary sorting is shown in Fig. 3. Fig. 3a shows the use of simple AND and OR gates to perform min and max operations with temporal coding, respectively. Fig. 3b illustrates the basic compare-and-swap unit for unary sorting. The input temporal signals are ranked in order at the output, with the larger values clustered at the bottom. The simplicity of unary compare-and-swap units offers opportunities in integration with SRM0-RNL neurons, leading to a more optimized design.

III Motivation and Opportunity

The existing SRM0-RNL neuron design assumes the worst-case scenarios. For an $n$-input neuron, the PC must accumulate all $n$ inputs, even if temporal spikes are not present for some inputs. This worst-case neuron design does not consider the biological aspect of neurons, i.e. it is de facto that neuron spikes are extremely sparse and only $0.1\%\sim 10\%$ of total neurons are actively spiking in any given compute cycle  [11, 20, 10]. Such a worst-case neuron design overprovisions hardware resources, incurring suboptimal efficiency.

Since the inputs of SRM0-RNL neurons are temporal coded, there exist opportunities to find the inputs with effective spikes using the unary logic in Fig. 3, and then use a more lightweight PC to aggregate these effective spikes.

In this work, we propose Catwalk to take advantage of this existing technique and optimize spike aggregation for more efficient SRM0-RNL neurons in TNNs. Our proposed Catwalk neuron model ingests all temporal spikes from pre-synaptic neurons and relocates them, so that all active spikes are clustered together. Then we can substitute the original full PC with a more lightweight version. As long as the cost of the spike relocation and the new PC is less than the original full PC, Catwalk offers hardware efficiency gains. Given the high neuronal sparsity within actual workloads, Catwalk should not cause significant accuracy concerns. More experimental work is needed to validate this.

IV Catwalk Neuron via Unary Top-K

In this section, we describe the hardware design of the proposed Catwalk neuron using unary top-k.

IV-A Catwalk* Neuron Microarchitecture*

We show the microarchitecture of the existing SRM0-RNL neuron and our Catwalk neuron with unary top k in Fig. 4. As illustrated in Fig. 2b, the key idea of Catwalk is to identify the valid temporal spikes, which simply means finding the sparse bit ones at each clock cycle from an input collection of bit ones and zeros. We coin this function as unary top k. As shown in Fig. 4a, existing SRM0-RNL neurons employ large PCs for accumulating response functions of input dendrites, requiring $n-1$ full adders for $n$ inputs [7]. Our Catwalk neuron simply replace the large PC in the dendrite with unary top-k and smaller PC, with no other changes to the soma and axon. As TNNs integrate multiple SRM0-RNL neurons into one TNN column [13, 12, 7], Catwalk is a plug-and-play component that contributes to overall improvements in TNN efficiency.

IV-B Unary Top-K

To implement unary top-k, we start with unary sorting in Fig. 3. Unary sorting has all inputs sorted and is very straightforward to find the top-k of all inputs. We show a few examples in Fig. 5, assuming that the outputs are in an ascending order from top to bottom. We evaluate two types of unary sorters: bitonic and optimal [2]. Bitonic sorters follow a structured bitonic pattern, while optimal sorters minimize the number of compare-and-swap units, achieving the lowest known count. We use Algorithm 1 to prune a given unary sorter and obtain the corresponding unary top-k selector.

According to Fig. 5, we have the following three observations. First different unary sorters can yield identical top-k results with varying pruning efficiency; for top-2, bitonic and optimal sorters prune equally, but for top-4, bitonic prunes more. Second, the final cost of unary top-k is independent of the cost reduction in compare-and-swap units. For both top-2 and top-4, the final costs depend on both the original cost and the cost reduction. Third, the higher the $k$, the higher the hardware cost. From top-2 to top-4, less compare-and-swap units can be pruned, as well with the half compare-and-swap units (blue in Fig. 5).

In general, we observe that optimal sorters yield better results, so we choose them for this work. However, a gap remains between unary sorting and unary top-k, as directly selecting the top k without full sorting could be even more resource-efficient. This work focuses on optimal sorting-based solutions, leaving optimal top-k selection for future research.

V Experimental Setup

We performed hardware evaluations using the NanGate45 standard cell library to obtain 45 nm CMOS implementation results and evaluation results. Design configurations of 16, 32, and 64 neuron inputs are chosen to enable comparisons across different scales. Synthesis carried out in Synopsys Design Compiler for three distinct design hierarchy configurations: (i) a stand-alone sorting/top-k stage, including unary bitonic sorters and optimal unary top-k, (ii) a sorting/top-k stage interfaced with a PC (a conventional design and a compact design), and (iii) bitonic sorting/optimal top-2 stage interfaced with a PC and augmented with a thresholding and firing unit, representing a SRM0-RNL neuron. At the final design hierarchy, all configurations are clocked at 400 MHz to ensure consistent timing assumptions.

Designs are then placed and routed using Cadence Innovus with the NanGate45 cell library. For these experiments, we again clock the designs at 400 MHz using a square floor plan with 70% utilization for each input size to provide a consistent basis for comparison.

VI Evaluation

This section evaluates the cost of our proposed Catwalk neuron from both theoretical and experimental aspects.

VI-A Gate Count Analysis

Given the crux of using a lightweight unary top-k selector and PC to replace the original large PC, and the corresponding large design space, we seek to understand the potential of Catwalk. More specifically, we examine the gate count of multiple unary top-k and dendrite input designs. We assume power-of-2 values for all $n$ (inputs) and $k$ (top-$k$ selections).

Fig. 6a shows the gate count of different unary top-k designs. We use optimal sorters to derive these unary top-k selectors. The combined stacked bar is the total gate count for the remaining compare-and-swap units after pruning.

We observe that pruning compare-and-swap units significantly reduces hardware costs. Additionally, removing gates from half compare-and-swap units provides a smaller, but still helpful, optimization. The gate saving trends for $n=\{16,32,64\}$ demonstrates the potential of unary top-k in reducing the dendrite cost, thus neuron cost, from a theoretical aspect. Fig. 6b shows the gate count for different dendrite designs. We observe that when $k=2$, unary top-k offers gains in gate count, while larger $k$ values do not. This is due to unary sorting-based unary top-k becomes more costly with larger $k$, as shown in Fig. 5.

VI-B Synthesis Results

VI-B1 Unary Top-K

We synthesize unary sorting and unary top-k based on optimal sorting, which has the minimum number of compare-and-swap units known to date [2]. Only $n=\{4,8,16,32,64\}$ are publicly available, and we leave the exploration of larger $n$ to future work. The unary top-k selectors are obtained using the same method as in Fig. 5. We show the results in Fig. 7. Fig. 7a and Fig. 7b show the synthesized area and power for different $n$ and $k$, and we observe graceful scaling when sweeping $n$ and $k$.

VI-B2 Dendrite

We also synthesize different dendrites, with results given in Fig. 8. To avoid an exploding design space, we do not explore all dendrite combinations, but focus on the most efficient and available options. As the sparsity in neurons can be as low as 0.1% [11, 20, 10], we consider $k=2$ sufficient for the input count $n=\{16,32,64\}$. We do not consider larger $n$, as no such optimal sorters are publicly available [2]. Note that with $k=2$, the PC for top-k and sorting is just one full adder, as shown in Fig. 4b.

Three observations can be made. First, unary top-k offers up to $1.17\times$ area savings over two PCs, aligning with theoretical gate count analysis. However, unary sorting does not consistently show better or worse area. Second, conventional PC (using an adder tree for accumulation) does not show worse area and power compared to compact PC (Fig. 4a). A conventional PC should have a larger cost in theory, but it is reasonable in the small scale that we are focusing on. Third, both top-k and sorting show significant reduction in power consumption. The leakage power of different designs remains similar, while top-k (and sorting) lowers the dynamic power significantly, boosting the power efficiency by up to $4.52\times$.

VI-B3 Neuron

We further synthesize the full neuron, including dendrite, soma, and axon, with results given in Fig. 9. The experimental setup here is identical to that in dendrite evaluation. Catwalk (Top-2 PC) improves area and power by $1.05\times$ and $1.35\times$ over the neuron with a compact PC, and by $1.05\times$ and $1.17\times$ over the sorting-based neuron. This aligns with insights from dendrite evaluation: while area reduction is limited, power improvement is significant.

VI-C Place and Route Results

In addition to synthesis, we also place and route all neurons, with the results in Table I. We observe that the leakage power of different design does not change too much, and Catwalk’s benefits mainly origins from the reduction in dynamic power. Compared with the compact PC-based neuron (existing SRM0-RNL neuron design [7]), the area of Catwalk is improved by $1.23\times$, $1.32\times$ and $1.39$; the power of Catwalk is improved by $1.38\times$, $1.67\times$ and $1.86\times$ for $n=16,32,64$. In general, more improvements are present with larger $n$. Then the improvements are slightly more compared to the conventional PC-based neuron. Also, within the evaluated range, Catwalk indeed shows both area and power improvements compared to sorting PC-based neurons, indicating the importance of opting for top-k over sorting, despite identical functionality.

VII Conclusion

This work identifies suboptimal efficiency of ramp-no-leak neurons in temporal neural networks, due to worse-case spike aggregation. We propose to relocate the temporal spikes via unary top-k to reduce the hardware cost. Unary top-k can be efficiently derived from unary sorting, with potential to reduce dendritic costs in neurons. Through the use of lightweight unary top-k and parallel counter, we show that the post place-and-route area and power can be improved by up to $1.39\times$ and $1.86\times$, respectively, compared to existing neurons.