FlowSE: Flow Matching-based Speech Enhancement

TL;DR

FlowSE introduces a flow matching-based speech enhancement method that achieves diffusion-model-like performance with significantly fewer function evaluations, reducing computational complexity without fine-tuning.

Contribution

This paper presents a novel conditional flow matching approach for speech enhancement that matches diffusion model performance with fewer function evaluations and no fine-tuning.

Findings

Achieved diffusion-level performance with NFE of 5

Matched diffusion model performance without fine-tuning

Reduced computational complexity in speech enhancement

Abstract

Diffusion probabilistic models have shown impressive performance for speech enhancement, but they typically require 25 to 60 function evaluations in the inference phase, resulting in heavy computational complexity. Recently, a fine-tuning method was proposed to correct the reverse process, which significantly lowered the number of function evaluations (NFE). Flow matching is a method to train continuous normalizing flows which model probability paths from known distributions to unknown distributions including those described by diffusion processes. In this paper, we propose a speech enhancement based on conditional flow matching. The proposed method achieved the performance comparable to those for the diffusion-based speech enhancement with the NFE of 60 when the NFE was 5, and showed similar performance with the diffusion model correcting the reverse process at the same NFE from 1 to 5 without additional fine tuning procedure. We also have shown that the corresponding diffusion model derived from the conditional probability path with a modified optimal transport conditional vector field demonstrated similar performances with the NFE of 5 without any fine-tuning procedure.