Transformer-based Automatic Speech Recognition of Formal and Colloquial Czech in MALACH Project

TL;DR

This paper investigates how self-supervised audio Transformers, specifically Wav2Vec 2.0, perform in transcribing colloquial Czech speech into formal transcripts, addressing language form disparities in ASR systems.

Contribution

It explores the impact of colloquial speech on Wav2Vec 2.0 ASR models and evaluates training strategies using formal and colloquial Czech forms.

Findings

Colloquial speech influences ASR performance significantly.

Training with both formal and colloquial data improves transcription accuracy.

Results demonstrate the model's ability to handle language form variations.

Abstract

Czech is a very specific language due to its large differences between the formal and the colloquial form of speech. While the formal (written) form is used mainly in official documents, literature, and public speeches, the colloquial (spoken) form is used widely among people in casual speeches. This gap introduces serious problems for ASR systems, especially when training or evaluating ASR models on datasets containing a lot of colloquial speech, such as the MALACH project. In this paper, we are addressing this problem in the light of a new paradigm in end-to-end ASR systems -- recently introduced self-supervised audio Transformers. Specifically, we are investigating the influence of colloquial speech on the performance of Wav2Vec 2.0 models and their ability to transcribe colloquial speech directly into formal transcripts. We are presenting results with both formal and colloquial forms in the training transcripts, language models, and evaluation transcripts.