Real-time processing of analog signals on accelerated neuromorphic hardware

TL;DR

This paper demonstrates a novel approach for real-time analog signal processing on neuromorphic hardware by directly injecting continuous signals, enabling efficient sensory processing and actuator control without digital conversions.

Contribution

It introduces the first direct analog input method for BrainScaleS-2 neuromorphic hardware, enabling fully on-chip sensory processing and actuation with high acceleration.

Findings

Successful real-time sound source localization and actuator alignment

First demonstration of continuous analog signal injection into BrainScaleS-2

Achieved 1000-fold acceleration in neuromorphic processing

Abstract

Sensory processing with neuromorphic systems is typically done by using either event-based sensors or translating input signals to spikes before presenting them to the neuromorphic processor. Here, we offer an alternative approach: direct analog signal injection eliminates superfluous and power-intensive analog-to-digital and digital-to-analog conversions, making it particularly suitable for efficient near-sensor processing. We demonstrate this by using the accelerated BrainScaleS-2 mixed-signal neuromorphic research platform and interfacing it directly to microphones and a servo-motor-driven actuator. Utilizing BrainScaleS-2's 1000-fold acceleration factor, we employ a spiking neural network to transform interaural time differences into a spatial code and thereby predict the location of sound sources. Our primary contributions are the first demonstrations of direct, continuous-valued sensor data injection into the analog compute units of the BrainScaleS-2 ASIC, and actuator control using its embedded microprocessors. This enables a fully on-chip processing pipeline$\unicode{x2014}$from sensory input handling, via spiking neural network processing to physical action. We showcase this by programming the system to localize and align a servo motor with the spatial direction of transient noise peaks in real-time.