Diffusion-based Generative Modeling with Discriminative Guidance for Streamable Speech Enhancement

TL;DR

This paper introduces a computationally efficient diffusion-based speech enhancement method using discriminative guidance, enabling real-time streaming with minimal latency and maintained performance.

Contribution

It proposes a novel approach combining discriminative scores with diffusion models for streaming speech enhancement, reducing computation and latency.

Findings

Significant reduction in RDP iterations needed for enhancement

Achieved 50 ms latency suitable for streaming applications

Maintained comparable performance to offline models

Abstract

Diffusion-based generative models (DGMs) have recently attracted attention in speech enhancement research (SE) as previous works showed a remarkable generalization capability. However, DGMs are also computationally intensive, as they usually require many iterations in the reverse diffusion process (RDP), making them impractical for streaming SE systems. In this paper, we propose to use discriminative scores from discriminative models in the first steps of the RDP. These discriminative scores require only one forward pass with the discriminative model for multiple RDP steps, thus greatly reducing computations. This approach also allows for performance improvements. We show that we can trade off between generative and discriminative capabilities as the number of steps with the discriminative score increases. Furthermore, we propose a novel streamable time-domain generative model with an algorithmic latency of 50 ms, which has no significant performance degradation compared to offline models.