PCNN: A Lightweight Parallel Conformer Neural Network for Efficient Monaural Speech Enhancement

TL;DR

This paper introduces PCNN, a lightweight parallel conformer neural network that combines CNN and Transformer components for efficient monaural speech enhancement, achieving superior performance with fewer parameters.

Contribution

The paper proposes a novel Parallel Conformer architecture that effectively unifies CNN and Transformer for speech enhancement, featuring a multi-branch dilated convolution and self-channel-time-frequency attention modules.

Findings

Outperforms state-of-the-art methods in most evaluation metrics.

Maintains the lowest model parameters among compared approaches.

Demonstrates improved local and global feature extraction.

Abstract

Convolutional neural networks (CNN) and Transformer have wildly succeeded in multimedia applications. However, more effort needs to be made to harmonize these two architectures effectively to satisfy speech enhancement. This paper aims to unify these two architectures and presents a Parallel Conformer for speech enhancement. In particular, the CNN and the self-attention (SA) in the Transformer are fully exploited for local format patterns and global structure representations. Based on the small receptive field size of CNN and the high computational complexity of SA, we specially designed a multi-branch dilated convolution (MBDC) and a self-channel-time-frequency attention (Self-CTFA) module. MBDC contains three convolutional layers with different dilation rates for the feature from local to non-local processing. Experimental results show that our method performs better than state-of-the-art methods in most evaluation criteria while maintaining the lowest model parameters.