Separating Long-Form Speech with Group-Wise Permutation Invariant Training

TL;DR

This paper introduces Group-PIT, a novel training scheme for long-form speech separation that improves efficiency and leverages long-span relationships, enhancing multi-talker conversation processing.

Contribution

The paper proposes Group-PIT, a new training method enabling effective long-form speech separation with reduced computational cost and improved handling of long-span utterance relationships.

Findings

Group-PIT improves separation performance on long speech inputs.

The approach reduces computational costs compared to traditional methods.

Effective in simulated meeting-style scenarios.

Abstract

Multi-talker conversational speech processing has drawn many interests for various applications such as meeting transcription. Speech separation is often required to handle overlapped speech that is commonly observed in conversation. Although the original utterancelevel permutation invariant training-based continuous speech separation approach has proven to be effective in various conditions, it lacks the ability to leverage the long-span relationship of utterances and is computationally inefficient due to the highly overlapped sliding windows. To overcome these drawbacks, we propose a novel training scheme named Group-PIT, which allows direct training of the speech separation models on the long-form speech with a low computational cost for label assignment. Two different speech separation approaches with Group-PIT are explored, including direct long-span speech separation and short-span speech separation with long-span tracking. The experiments on the simulated meeting-style data demonstrate the effectiveness of our proposed approaches, especially in dealing with a very long speech input.