Monaural Speech Enhancement Using a Multi-Branch Temporal Convolutional Network

TL;DR

This paper introduces a multi-branch temporal convolutional network (MB-TCN) for monaural speech enhancement, effectively capturing long-term temporal dependencies with lower complexity and outperforming existing methods in speech quality and intelligibility.

Contribution

The paper proposes a novel multi-branch TCN architecture that combines split-transform-aggregate design with residual dilated CNNs for improved speech enhancement.

Findings

MB-TCN outperforms ResLSTMs, TCNs, and dense CNNs in speech quality and intelligibility.

The model achieves superior results on five objective metrics compared to state-of-the-art methods.

MB-TCN demonstrates high parameter efficiency and robustness in speech enhancement tasks.

Abstract

Deep learning has achieved substantial improvement on single-channel speech enhancement tasks. However, the performance of multi-layer perceptions (MLPs)-based methods is limited by the ability to capture the long-term effective history information. The recurrent neural networks (RNNs), e.g., long short-term memory (LSTM) model, are able to capture the long-term temporal dependencies, but come with the issues of the high latency and the complexity of training.To address these issues, the temporal convolutional network (TCN) was proposed to replace the RNNs in various sequence modeling tasks. In this paper we propose a novel TCN model that employs multi-branch structure, called multi-branch TCN (MB-TCN), for monaural speech enhancement.The MB-TCN exploits split-transform-aggregate design, which is expected to obtain strong representational power at a low computational complexity.Inspired by the TCN, the MB-TCN model incorporates one dimensional causal dilated CNN and residual learning to expand receptive fields for capturing long-term temporal contextual information.Our extensive experimental investigation suggests that the MB-TCNs outperform the residual long short-term memory networks (ResLSTMs), temporal convolutional networks (TCNs), and the CNN networks that employ dense aggregations in terms of speech intelligibility and quality, while providing superior parameter efficiency. Furthermore, our experimental results demonstrate that our proposed MB-TCN model is able to outperform multiple state-of-the-art deep learning-based speech enhancement methods in terms of five widely used objective metrics.