Multiple Choice Learning for Efficient Speech Separation with Many Speakers

TL;DR

This paper explores using Multiple Choice Learning (MCL) for speech separation, showing it matches Permutation Invariant Training (PIT) performance while offering computational benefits and potential for handling variable speakers and unsupervised tasks.

Contribution

The paper introduces MCL as an alternative to PIT for speech separation, demonstrating its effectiveness and advantages on standard benchmarks.

Findings

MCL achieves comparable performance to PIT on WSJ0-mix and LibriMix.

MCL offers computational advantages over PIT.

MCL can be extended to variable speaker counts and unsupervised learning.

Abstract

Training speech separation models in the supervised setting raises a permutation problem: finding the best assignation between the model predictions and the ground truth separated signals. This inherently ambiguous task is customarily solved using Permutation Invariant Training (PIT). In this article, we instead consider using the Multiple Choice Learning (MCL) framework, which was originally introduced to tackle ambiguous tasks. We demonstrate experimentally on the popular WSJ0-mix and LibriMix benchmarks that MCL matches the performances of PIT, while being computationally advantageous. This opens the door to a promising research direction, as MCL can be naturally extended to handle a variable number of speakers, or to tackle speech separation in the unsupervised setting.