Neuromorphic Keyword Spotting with Pulse Density Modulation MEMS Microphones

TL;DR

This paper introduces a direct connection between Pulse Density Modulation microphones and Spiking Neural Networks for keyword spotting, reducing computational costs and achieving high accuracy with potential for low energy consumption.

Contribution

It proposes a novel direct microphone-to-SNN pipeline leveraging PDM microphones, eliminating intermediate stages and enhancing energy efficiency in neuromorphic keyword spotting.

Findings

Achieved 91.54% accuracy on GSC dataset.

Surpassed state-of-the-art on SSC dataset.

Observed network sparsity indicating low energy potential.

Abstract

The Keyword Spotting (KWS) task involves continuous audio stream monitoring to detect predefined words, requiring low energy devices for continuous processing. Neuromorphic devices effectively address this energy challenge. However, the general neuromorphic KWS pipeline, from microphone to Spiking Neural Network (SNN), entails multiple processing stages. Leveraging the popularity of Pulse Density Modulation (PDM) microphones in modern devices and their similarity to spiking neurons, we propose a direct microphone-to-SNN connection. This approach eliminates intermediate stages, notably reducing computational costs. The system achieved an accuracy of 91.54\% on the Google Speech Command (GSC) dataset, surpassing the state-of-the-art for the Spiking Speech Command (SSC) dataset which is a bio-inspired encoded GSC. Furthermore, the observed sparsity in network activity and connectivity indicates potential for remarkably low energy consumption in a neuromorphic device implementation.