Improving the Training Recipe for a Robust Conformer-based Hybrid Model

TL;DR

This paper introduces a new speaker adaptation method called Weighted-Simple-Add for conformer-based ASR models, achieving significant WER improvements and optimizing the training recipe for efficiency and performance.

Contribution

It proposes the Weighted-Simple-Add method for speaker adaptation and enhances the conformer training recipe for better accuracy and efficiency.

Findings

3.5% and 4.5% relative WER reduction on Hub5'00 and Hub5'01

11% relative WER improvement on Switchboard 300h

34% reduction in model parameters for efficiency

Abstract

Speaker adaptation is important to build robust automatic speech recognition (ASR) systems. In this work, we investigate various methods for speaker adaptive training (SAT) based on feature-space approaches for a conformer-based acoustic model (AM) on the Switchboard 300h dataset. We propose a method, called Weighted-Simple-Add, which adds weighted speaker information vectors to the input of the multi-head self-attention module of the conformer AM. Using this method for SAT, we achieve 3.5% and 4.5% relative improvement in terms of WER on the CallHome part of Hub5'00 and Hub5'01 respectively. Moreover, we build on top of our previous work where we proposed a novel and competitive training recipe for a conformer-based hybrid AM. We extend and improve this recipe where we achieve 11% relative improvement in terms of word-error-rate (WER) on Switchboard 300h Hub5'00 dataset. We also make this recipe efficient by reducing the total number of parameters by 34% relative.