Neuromorphic Computing: A Theoretical Framework for Time, Space, and Energy Scaling

TL;DR

This paper introduces a theoretical framework for neuromorphic computing, analyzing its time, space, and energy scaling, and compares it to conventional systems, highlighting its potential for energy efficiency especially with sparse activity.

Contribution

It provides a novel computational framework for analyzing NMC algorithms and architectures, revealing unique energy scaling properties and suitability of certain algorithms for neuromorphic systems.

Findings

NMC has comparable time and space tradeoffs to conventional systems.

Energy costs in NMC are event-driven and depend on activity trace.

Sparse and decaying activity can lead to asymptotically improved energy scaling.

Abstract

Neuromorphic computing (NMC) is increasingly viewed as a low-power alternative to conventional von Neumann architectures such as central processing units (CPUs) and graphics processing units (GPUs), however the computational value proposition has been difficult to define precisely. Here, we propose a computational framework for analyzing NMC algorithms and architectures. Using this framework, we demonstrate that NMC can be analyzed as general-purpose and programmable even though it differs considerably from a conventional stored-program architecture. We show that the time and space scaling of idealized NMC has comparable time and footprint tradeoffs that align with that of a theoretically infinite processor conventional system. In contrast, energy scaling for NMC is significantly different than conventional systems, as NMC energy costs are event-driven. Using this framework, we show that while energy in conventional systems is largely determined by the scheduled operations determined by the structural algorithm graph, the energy of neuromorphic systems scales with the activity of the algorithm, that is the activity trace of the algorithm graph. Without making strong assumptions on NMC or conventional costs, we demonstrate which neuromorphic algorithm formulations can exhibit asymptotically improved energy scaling when activity is sparse and decaying over time. We further use these results to identify which broad algorithm families are more or less suitable for NMC approaches.