NeuroAI Temporal Neural Networks (NeuTNNs): Microarchitecture and Design Framework for Specialized Neuromorphic Processing Units

TL;DR

This paper introduces NeuTNNs, a new class of temporal neural networks inspired by neuroscience, with a design framework that enhances performance and efficiency for neuromorphic hardware applications.

Contribution

It proposes NeuTNNs incorporating neuroscience findings and develops NeuTNNGen, a PyTorch-based tool suite for designing application-specific NeuTNNs with improved efficiency.

Findings

NeuTNNs outperform previous TNNs in accuracy and efficiency.

Synaptic pruning reduces hardware costs by 30-50%.

NeuTNNGen successfully designs models for diverse applications.

Abstract

Leading experts from both communities have suggested the need to (re)connect research in neuroscience and artificial intelligence (AI) to accelerate the development of next-generation AI innovations. They term this convergence as NeuroAI. Previous research has established temporal neural networks (TNNs) as a promising neuromorphic approach toward biological intelligence and efficiency. We fully embrace NeuroAI and propose a new category of TNNs we call NeuroAI TNNs (NeuTNNs) with greater capability and hardware efficiency by adopting neuroscience findings, including a neuron model with active dendrites and a hierarchy of distal and proximal segments. This work introduces a PyTorch-to-layout tool suite (NeuTNNGen) to design application-specific NeuTNNs. Compared to previous TNN designs, NeuTNNs achieve superior performance and efficiency. We demonstrate NeuTNNGen's capabilities using three example applications: 1) UCR time series benchmarks, 2) MNIST design exploration, and 3) Place Cells design for neocortical reference frames. We also explore using synaptic pruning to further reduce synapse counts and hardware costs by 30-50% while maintaining model precision across diverse sensory modalities. NeuTNNGen can facilitate the design of application-specific energy-efficient NeuTNNs for the next generation of NeuroAI computing systems.