Generic Speech Enhancement with Self-Supervised Representation Space Loss

TL;DR

This paper introduces a self-supervised representation space loss for speech enhancement, enabling a generic model that improves multiple downstream speech tasks without task-specific tuning.

Contribution

The study proposes a novel training criterion based on self-supervised feature space distance, enhancing generalization across various speech enhancement tasks.

Findings

Improves performance on multiple speech tasks

Maintains perceptual quality of enhanced speech

Enables task-agnostic speech enhancement models

Abstract

Single-channel speech enhancement is utilized in various tasks to mitigate the effect of interfering signals. Conventionally, to ensure the speech enhancement performs optimally, the speech enhancement has needed to be tuned for each task. Thus, generalizing speech enhancement models to unknown downstream tasks has been challenging. This study aims to construct a generic speech enhancement front-end that can improve the performance of back-ends to solve multiple downstream tasks. To this end, we propose a novel training criterion that minimizes the distance between the enhanced and the ground truth clean signal in the feature representation domain of self-supervised learning models. Since self-supervised learning feature representations effectively express high-level speech information useful for solving various downstream tasks, the proposal is expected to make speech enhancement models preserve such information. Experimental validation demonstrates that the proposal improves the performance of multiple speech tasks while maintaining the perceptual quality of the enhanced signal.