Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matching

TL;DR

Mask2Flow-TSE introduces a two-stage target speaker extraction framework combining discriminative masking and flow matching, achieving high-quality speech separation efficiently and with fewer parameters.

Contribution

The paper presents a novel two-stage TSE method that integrates discriminative and generative paradigms, enabling effective and efficient target speech extraction.

Findings

Achieves comparable performance to existing generative methods.

Operates with approximately 85 million parameters.

Provides high-quality speech reconstruction in a single inference step.

Abstract

Target speaker extraction (TSE) extracts the target speaker's voice from overlapping speech mixtures given a reference utterance. Existing approaches typically fall into two categories: discriminative and generative. Discriminative methods apply time-frequency masking for fast inference but often over-suppress the target signal, while generative methods synthesize high-quality speech at the cost of numerous iterative steps. We propose Mask2Flow-TSE, a two-stage framework combining the strengths of both paradigms. The first stage applies discriminative masking for coarse separation, and the second stage employs flow matching to refine the output toward target speech. Unlike generative approaches that synthesize speech from Gaussian noise, our method starts from the masked spectrogram, enabling high-quality reconstruction in a single inference step. Experiments show that Mask2Flow-TSE achieves comparable performance to existing generative TSE methods with approximately 85M parameters.