A Microarchitecture Implementation Framework for Online Learning with Temporal Neural Networks

TL;DR

This paper presents a CMOS microarchitecture framework for Temporal Neural Networks that enables efficient online learning and inference, suitable for edge sensory processing units, with detailed gate-level implementations and performance analysis.

Contribution

It introduces a microarchitecture framework for TNNs with gate-level implementations and scaling equations, facilitating efficient online learning in CMOS technology.

Findings

Post-synthesis CMOS results demonstrate practical feasibility.

The microarchitecture supports online incremental learning.

Scaling equations enable design assessment for various TNN configurations.

Abstract

Temporal Neural Networks (TNNs) are spiking neural networks that use time as a resource to represent and process information, similar to the mammalian neocortex. In contrast to compute-intensive deep neural networks that employ separate training and inference phases, TNNs are capable of extremely efficient online incremental/continual learning and are excellent candidates for building edge-native sensory processing units. This work proposes a microarchitecture framework for implementing TNNs using standard CMOS. Gate-level implementations of three key building blocks are presented: 1) multi-synapse neurons, 2) multi-neuron columns, and 3) unsupervised and supervised online learning algorithms based on Spike Timing Dependent Plasticity (STDP). The proposed microarchitecture is embodied in a set of characteristic scaling equations for assessing the gate count, area, delay and power for any TNN design. Post-synthesis results (in 45nm CMOS) for the proposed designs are presented, and their online incremental learning capability is demonstrated.