Efficient Video and Audio processing with Loihi 2

TL;DR

This paper presents Loihi 2, a neuromorphic processor that enhances video and audio processing efficiency and latency through innovative neuron models and event-driven spikes, outperforming traditional solutions.

Contribution

Introduction of Loihi 2 with new neuromorphic features like sigma-delta and resonate-and-fire neurons for improved multimedia processing.

Findings

Orders of magnitude energy-delay-product improvements

Enhanced efficiency in video and audio neural networks

Significant latency reductions in signal processing tasks

Abstract

Loihi 2 is an asynchronous, brain-inspired research processor that generalizes several fundamental elements of neuromorphic architecture, such as stateful neuron models communicating with event-driven spikes, in order to address limitations of the first generation Loihi. Here we explore and characterize some of these generalizations, such as sigma-delta encapsulation, resonate-and-fire neurons, and integer-valued spikes, as applied to standard video, audio, and signal processing tasks. We find that these new neuromorphic approaches can provide orders of magnitude gains in combined efficiency and latency (energy-delay-product) for feed-forward and convolutional neural networks applied to video, audio denoising, and spectral transforms compared to state-of-the-art solutions.