Effects of Layer Freezing on Transferring a Speech Recognition System to Under-resourced Languages

TL;DR

This study explores how freezing layers in a speech recognition model affects transfer learning performance for under-resourced languages, demonstrating that even minimal freezing can significantly improve results.

Contribution

It systematically evaluates layer freezing schemes in transfer learning for speech recognition, highlighting the benefits of partial freezing in low-resource scenarios.

Findings

Freezing one layer significantly improves transfer performance.

Layer freezing schemes outperform training from scratch.

Partial freezing is effective for under-resourced languages.

Abstract

In this paper, we investigate the effect of layer freezing on the effectiveness of model transfer in the area of automatic speech recognition. We experiment with Mozilla's DeepSpeech architecture on German and Swiss German speech datasets and compare the results of either training from scratch vs. transferring a pre-trained model. We compare different layer freezing schemes and find that even freezing only one layer already significantly improves results.