Speech Enhancement using Separable Polling Attention and Global Layer Normalization followed with PReLU

TL;DR

This paper proposes a compact neural network model for single-channel speech enhancement that outperforms larger models by introducing separable polling attention, global layer normalization with PReLU, and replacing BLSTM with Conv2d, significantly reducing size and improving performance.

Contribution

The paper introduces techniques to shrink the PHASEN model while enhancing its speech enhancement performance, including separable polling attention and normalization improvements.

Findings

Model size reduced from 33M to 5M parameters.

Performance improved with higher CSIG, PESQ, and COVL scores.

Achieved state-of-the-art results with a smaller model.

Abstract

Single channel speech enhancement is a challenging task in speech community. Recently, various neural networks based methods have been applied to speech enhancement. Among these models, PHASEN and T-GSA achieve state-of-the-art performances on the publicly opened VoiceBank+DEMAND corpus. Both of the models reach the COVL score of 3.62. PHASEN achieves the highest CSIG score of 4.21 while T-GSA gets the highest PESQ score of 3.06. However, both of these two models are very large. The contradiction between the model performance and the model size is hard to reconcile. In this paper, we introduce three kinds of techniques to shrink the PHASEN model and improve the performance. Firstly, seperable polling attention is proposed to replace the frequency transformation blocks in PHASEN. Secondly, global layer normalization followed with PReLU is used to replace batch normalization followed with ReLU. Finally, BLSTM in PHASEN is replaced with Conv2d operation and the phase stream is simplified. With all these modifications, the size of the PHASEN model is shrunk from 33M parameters to 5M parameters, while the performance on VoiceBank+DEMAND is improved to the CSIG score of 4.30, the PESQ score of 3.07 and the COVL score of 3.73.