Joint domain adaptation and speech bandwidth extension using time-domain GANs for speaker verification

TL;DR

This paper introduces a joint learning approach using time-domain GANs to simultaneously perform domain adaptation and bandwidth extension for speech, improving speaker verification accuracy across different acoustic conditions.

Contribution

It proposes a novel joint training framework for domain adaptation and bandwidth extension using GANs, with both parallel and non-parallel data, outperforming separate models.

Findings

22% relative EER improvement on SRE16

Effective use of both paired and unpaired data

First evidence that joint learning surpasses individual tasks

Abstract

Speech systems developed for a particular choice of acoustic domain and sampling frequency do not translate easily to others. The usual practice is to learn domain adaptation and bandwidth extension models independently. Contrary to this, we propose to learn both tasks together. Particularly, we learn to map narrowband conversational telephone speech to wideband microphone speech. We developed parallel and non-parallel learning solutions which utilize both paired and unpaired data. First, we first discuss joint and disjoint training of multiple generative models for our tasks. Then, we propose a two-stage learning solution where we use a pre-trained domain adaptation system for pre-processing in bandwidth extension training. We evaluated our schemes on a Speaker Verification downstream task. We used the JHU-MIT experimental setup for NIST SRE21, which comprises SRE16, SRE-CTS Superset and SRE21. Our results provide the first evidence that learning both tasks is better than learning just one. On SRE16, our best system achieves 22% relative improvement in Equal Error Rate w.r.t. a direct learning baseline and 8% w.r.t. a strong bandwidth expansion system.