End-to-End Model for Speech Enhancement by Consistent Spectrogram Masking

TL;DR

This paper introduces a novel consistency spectrogram masking technique for speech enhancement that enforces spectrogram consistency, leading to faster training and improved speech quality.

Contribution

It proposes a simple yet effective consistency constraint in spectrogram masking, addressing issues of inconsistency and artifacts in phase-aware speech enhancement models.

Findings

CSM accelerates model training.

Significant improvements in speech quality.

Enhanced robustness of speech enhancement.

Abstract

Recently, phase processing is attracting increasinginterest in speech enhancement community. Some researchersintegrate phase estimations module into speech enhancementmodels by using complex-valued short-time Fourier transform(STFT) spectrogram based training targets, e.g. Complex RatioMask (cRM) [1]. However, masking on spectrogram would violentits consistency constraints. In this work, we prove that theinconsistent problem enlarges the solution space of the speechenhancement model and causes unintended artifacts. ConsistencySpectrogram Masking (CSM) is proposed to estimate the complexspectrogram of a signal with the consistency constraint in asimple but not trivial way. The experiments comparing ourCSM based end-to-end model with other methods are conductedto confirm that the CSM accelerate the model training andhave significant improvements in speech quality. From ourexperimental results, we assured that our method could enha