Towards Ultra-Low-Power Neuromorphic Speech Enhancement with Spiking-FullSubNet

TL;DR

This paper introduces Spiking-FullSubNet, an ultra-low-power neuromorphic speech enhancement system that leverages a novel spiking neural network architecture and human auditory-inspired frequency partitioning to outperform state-of-the-art methods in quality and energy efficiency.

Contribution

The work presents a novel spiking neural network architecture with a dynamic neuron model and a frequency partitioning method, achieving superior speech enhancement with ultra-low power consumption.

Findings

Outperforms state-of-the-art methods in speech quality and energy efficiency.

Wins the Intel N-DNS Challenge (Algorithmic Track).

Demonstrates effectiveness of neuromorphic approach for edge speech enhancement.

Abstract

Speech enhancement is critical for improving speech intelligibility and quality in various audio devices. In recent years, deep learning-based methods have significantly improved speech enhancement performance, but they often come with a high computational cost, which is prohibitive for a large number of edge devices, such as headsets and hearing aids. This work proposes an ultra-low-power speech enhancement system based on the brain-inspired spiking neural network (SNN) called Spiking-FullSubNet. Spiking-FullSubNet follows a full-band and sub-band fusioned approach to effectively capture both global and local spectral information. To enhance the efficiency of computationally expensive sub-band modeling, we introduce a frequency partitioning method inspired by the sensitivity profile of the human peripheral auditory system. Furthermore, we introduce a novel spiking neuron model that can dynamically control the input information integration and forgetting, enhancing the multi-scale temporal processing capability of SNN, which is critical for speech denoising. Experiments conducted on the recent Intel Neuromorphic Deep Noise Suppression (N-DNS) Challenge dataset show that the Spiking-FullSubNet surpasses state-of-the-art methods by large margins in terms of both speech quality and energy efficiency metrics. Notably, our system won the championship of the Intel N-DNS Challenge (Algorithmic Track), opening up a myriad of opportunities for ultra-low-power speech enhancement at the edge. Our source code and model checkpoints are publicly available at https://github.com/haoxiangsnr/spiking-fullsubnet.