On the Impact of Quantization and Pruning of Self-Supervised Speech Models for Downstream Speech Recognition Tasks "In-the-Wild''

TL;DR

This paper investigates how quantization and pruning affect the performance of self-supervised speech models, especially under challenging real-world conditions like noise and reverberation, to enable efficient deployment on resource-limited devices.

Contribution

It provides an analysis of the impact of model compression techniques on self-supervised speech models in diverse and challenging acoustic environments.

Findings

Quantization and pruning degrade speech recognition accuracy under noisy conditions.

Model compression effects vary with different types of acoustic distortions.

Results inform the design of resource-efficient speech recognition systems for real-world use.

Abstract

Recent advances with self-supervised learning have allowed speech recognition systems to achieve state-of-the-art (SOTA) word error rates (WER) while requiring only a fraction of the labeled training data needed by its predecessors. Notwithstanding, while such models achieve SOTA performance in matched train/test conditions, their performance degrades substantially when tested in unseen conditions. To overcome this problem, strategies such as data augmentation and/or domain shift training have been explored. Available models, however, are still too large to be considered for edge speech applications on resource-constrained devices, thus model compression tools are needed. In this paper, we explore the effects that train/test mismatch conditions have on speech recognition accuracy based on compressed self-supervised speech models. In particular, we report on the effects that parameter quantization and model pruning have on speech recognition accuracy based on the so-called robust wav2vec 2.0 model under noisy, reverberant, and noise-plus-reverberation conditions.