A deep representation learning speech enhancement method using $\beta$-VAE

TL;DR

This paper introduces a novel $eta$-VAE-based speech enhancement method that improves latent representation disentanglement, enhances speech quality, and reduces model complexity compared to previous PVAE approaches.

Contribution

The paper proposes a $eta$-VAE strategy that enhances representation learning in PVAE, overcoming the disentanglement-reconstruction trade-off and optimizing DNN structure.

Findings

Better speech and noise latent representations achieved

Higher scale-invariant SNR and speech quality

Reduced number of training parameters

Abstract

In previous work, we proposed a variational autoencoder-based (VAE) Bayesian permutation training speech enhancement (SE) method (PVAE) which indicated that the SE performance of the traditional deep neural network-based (DNN) method could be improved by deep representation learning (DRL). Based on our previous work, we in this paper propose to use $\beta$-VAE to further improve PVAE's ability of representation learning. More specifically, our $\beta$-VAE can improve PVAE's capacity of disentangling different latent variables from the observed signal without the trade-off problem between disentanglement and signal reconstruction. This trade-off problem widely exists in previous $\beta$-VAE algorithms. Unlike the previous $\beta$-VAE algorithms, the proposed $\beta$-VAE strategy can also be used to optimize the DNN's structure. This means that the proposed method can not only improve PVAE's SE performance but also reduce the number of PVAE training parameters. The experimental results show that the proposed method can acquire better speech and noise latent representation than PVAE. Meanwhile, it also obtains a higher scale-invariant signal-to-distortion ratio, speech quality, and speech intelligibility.