Cross-lingual Knowledge Transfer and Iterative Pseudo-labeling for Low-Resource Speech Recognition with Transducers

TL;DR

This paper presents a method to improve low-resource speech recognition by combining cross-lingual transfer and iterative pseudo-labeling, significantly reducing error rates in neural transducer ASR systems.

Contribution

It introduces a novel approach that combines cross-lingual knowledge transfer with iterative pseudo-labeling to enhance low-resource speech recognition accuracy.

Findings

18% reduction in word error rate using transcripts from hybrid systems

35% reduction in error rate with combined transfer and pseudo-labeling

Neural transducer system outperforms hybrid systems in low-resource settings

Abstract

Voice technology has become ubiquitous recently. However, the accuracy, and hence experience, in different languages varies significantly, which makes the technology not equally inclusive. The availability of data for different languages is one of the key factors affecting accuracy, especially in training of all-neural end-to-end automatic speech recognition systems. Cross-lingual knowledge transfer and iterative pseudo-labeling are two techniques that have been shown to be successful for improving the accuracy of ASR systems, in particular for low-resource languages, like Ukrainian. Our goal is to train an all-neural Transducer-based ASR system to replace a DNN-HMM hybrid system with no manually annotated training data. We show that the Transducer system trained using transcripts produced by the hybrid system achieves 18% reduction in terms of word error rate. However, using a combination of cross-lingual knowledge transfer from related languages and iterative pseudo-labeling, we are able to achieve 35% reduction of the error rate.