Guided Training: A Simple Method for Single-channel Speaker Separation

TL;DR

This paper introduces a straightforward training method for single-channel speaker separation that uses a short guide speech at the beginning of the mixture to help the LSTM model identify and separate the target speaker.

Contribution

The paper proposes a novel guided training strategy that leverages a short target speech segment to address permutation issues in speaker separation with LSTM models.

Findings

Effective in improving speaker separation performance

Utilizes sequence modeling capabilities of LSTM

Simplifies training process for speaker separation

Abstract

Deep learning has shown a great potential for speech separation, especially for speech and non-speech separation. However, it encounters permutation problem for multi-speaker separation where both target and interference are speech. Permutation Invariant training (PIT) was proposed to solve this problem by permuting the order of the multiple speakers. Another way is to use an anchor speech, a short speech of the target speaker, to model the speaker identity. In this paper, we propose a simple strategy to train a long short-term memory (LSTM) model to solve the permutation problem in speaker separation. Specifically, we insert a short speech of target speaker at the beginning of a mixture as guide information. So, the first appearing speaker is defined as the target. Due to the powerful capability on sequence modeling, LSTM can use its memory cells to track and separate target speech from interfering speech. Experimental results show that the proposed training strategy is effective for speaker separation.