ctPuLSE: Close-Talk, and Pseudo-Label Based Far-Field, Speech Enhancement

TL;DR

This paper introduces ctPuLSE, a novel training approach for far-field speech enhancement that leverages close-talk speech as pseudo-labels to improve real-world generalization of enhancement models.

Contribution

It proposes a new method to train far-field speech enhancement models directly on real data using pseudo-labels generated from close-talk speech enhancement.

Findings

ctPuLSE produces high-quality pseudo-labels.

Models trained with ctPuLSE generalize well to real data.

Significant improvement over traditional supervised methods.

Abstract

The current dominant approach for neural speech enhancement is via purely-supervised deep learning on simulated pairs of far-field noisy-reverberant speech (i.e., mixtures) and clean speech. The trained models, however, often exhibit limited generalizability to real-recorded mixtures. To deal with this, this paper investigates training enhancement models directly on real mixtures. However, a major difficulty challenging this approach is that, since the clean speech of real mixtures is unavailable, there lacks a good supervision for real mixtures. In this context, assuming that a training set consisting of real-recorded pairs of close-talk and far-field mixtures is available, we propose to address this difficulty via close-talk speech enhancement, where an enhancement model is first trained on simulated mixtures to enhance real-recorded close-talk mixtures and the estimated close-talk speech can then be utilized as a supervision (i.e., pseudo-label) for training far-field speech enhancement models directly on the paired real-recorded far-field mixtures. We name the proposed system ctPuLSE. Evaluation results on the popular CHiME-4 dataset show that ctPuLSE can derive high-quality pseudo-labels and yield far-field speech enhancement models with strong generalizability to real data.