Target Speaker Extraction with Curriculum Learning

TL;DR

This paper introduces a curriculum learning approach for target speaker extraction that gradually increases training difficulty, significantly improving performance over baseline models on the Libri2talker dataset.

Contribution

It proposes novel curriculum learning strategies tailored for target speaker extraction, enhancing model training by systematically increasing data complexity.

Findings

CL strategies improved performance by about 1 dB

Models trained with CL outperformed baseline models

Curriculum design effectively exposes models to more challenging scenarios

Abstract

This paper presents a novel approach to target speaker extraction (TSE) using Curriculum Learning (CL) techniques, addressing the challenge of distinguishing a target speaker's voice from a mixture containing interfering speakers. For efficient training, we propose designing a curriculum that selects subsets of increasing complexity, such as increasing similarity between target and interfering speakers, and that selects training data strategically. Our CL strategies include both variants using predefined difficulty measures (e.g. gender, speaker similarity, and signal-to-distortion ratio) and ones using the TSE's standard objective function, each designed to expose the model gradually to more challenging scenarios. Comprehensive testing on the Libri2talker dataset demonstrated that our CL strategies for TSE improved the performance, and the results markedly exceeded baseline models without CL about 1 dB.