PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Network

TL;DR

PHASEN is a novel speech enhancement neural network that effectively models phase and harmonic information, leading to significant improvements over previous methods in speech quality and noise reduction.

Contribution

This work introduces a two-stream DNN architecture with frequency transformation blocks for phase and harmonic-aware speech enhancement, a novel approach compared to prior amplitude-focused methods.

Findings

Achieves 1.76dB SDR improvement on AVSpeech + AudioSet dataset.

Outperforms Google's network significantly on the same dataset.

Surpasses previous methods on Voice Bank + DEMAND dataset across multiple metrics.

Abstract

Time-frequency (T-F) domain masking is a mainstream approach for single-channel speech enhancement. Recently, focuses have been put to phase prediction in addition to amplitude prediction. In this paper, we propose a phase-and-harmonics-aware deep neural network (DNN), named PHASEN, for this task. Unlike previous methods that directly use a complex ideal ratio mask to supervise the DNN learning, we design a two-stream network, where amplitude stream and phase stream are dedicated to amplitude and phase prediction. We discover that the two streams should communicate with each other, and this is crucial to phase prediction. In addition, we propose frequency transformation blocks to catch long-range correlations along the frequency axis. The visualization shows that the learned transformation matrix spontaneously captures the harmonic correlation, which has been proven to be helpful for T-F spectrogram reconstruction. With these two innovations, PHASEN acquires the ability to handle detailed phase patterns and to utilize harmonic patterns, getting 1.76dB SDR improvement on AVSpeech + AudioSet dataset. It also achieves significant gains over Google's network on this dataset. On Voice Bank + DEMAND dataset, PHASEN outperforms previous methods by a large margin on four metrics.