VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking

TL;DR

This paper introduces VoiceFilter, a neural network-based system that isolates a target speaker's voice from multi-speaker audio using a reference signal, improving speech recognition accuracy.

Contribution

It proposes a novel speaker-conditioned spectrogram masking approach combining speaker embeddings and neural networks for targeted voice separation.

Findings

Reduces speech recognition WER on multi-speaker signals

Maintains minimal WER degradation on single-speaker signals

Demonstrates effectiveness of speaker-conditioned masking

Abstract

In this paper, we present a novel system that separates the voice of a target speaker from multi-speaker signals, by making use of a reference signal from the target speaker. We achieve this by training two separate neural networks: (1) A speaker recognition network that produces speaker-discriminative embeddings; (2) A spectrogram masking network that takes both noisy spectrogram and speaker embedding as input, and produces a mask. Our system significantly reduces the speech recognition WER on multi-speaker signals, with minimal WER degradation on single-speaker signals.