Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-training

TL;DR

This paper introduces a self-supervised pre-training method to stabilize label assignment in speech separation models, improving convergence speed and performance.

Contribution

It proposes a novel self-supervised pre-training approach specifically designed to stabilize label assignment in speech separation training.

Findings

Self-supervised pre-training significantly improves speech separation performance.

The method stabilizes label assignment, leading to faster convergence.

Results are consistent across different models and datasets.

Abstract

Speech separation has been well developed, with the very successful permutation invariant training (PIT) approach, although the frequent label assignment switching happening during PIT training remains to be a problem when better convergence speed and achievable performance are desired. In this paper, we propose to perform self-supervised pre-training to stabilize the label assignment in training the speech separation model. Experiments over several types of self-supervised approaches, several typical speech separation models and two different datasets showed that very good improvements are achievable if a proper self-supervised approach is chosen.