Metric-oriented Speech Enhancement using Diffusion Probabilistic Model

TL;DR

This paper introduces MOSE, a speech enhancement method that uses diffusion probabilistic models and a metric-oriented training strategy to directly optimize evaluation metrics like PESQ, leading to improved performance.

Contribution

It proposes a novel metric-oriented training framework for diffusion-based speech enhancement, aligning training objectives with evaluation metrics.

Findings

MOSE outperforms baseline methods across all evaluation metrics.

The actor-critic framework effectively guides the reverse process towards metric improvement.

Metric-oriented training significantly enhances speech quality and intelligibility.

Abstract

Deep neural network based speech enhancement technique focuses on learning a noisy-to-clean transformation supervised by paired training data. However, the task-specific evaluation metric (e.g., PESQ) is usually non-differentiable and can not be directly constructed in the training criteria. This mismatch between the training objective and evaluation metric likely results in sub-optimal performance. To alleviate it, we propose a metric-oriented speech enhancement method (MOSE), which leverages the recent advances in the diffusion probabilistic model and integrates a metric-oriented training strategy into its reverse process. Specifically, we design an actor-critic based framework that considers the evaluation metric as a posterior reward, thus guiding the reverse process to the metric-increasing direction. The experimental results demonstrate that MOSE obviously benefits from metric-oriented training and surpasses the generative baselines in terms of all evaluation metrics.