Exploring Spoken Language Identification Strategies for Automatic Transcription of Multilingual Broadcast and Institutional Speech

TL;DR

This study proposes a cascaded speaker diarization and language identification system for multilingual broadcast speech, demonstrating significant improvements in language error rates and speech recognition accuracy in real-world scenarios.

Contribution

It introduces a novel cascaded approach combining speaker diarization with language identification, tailored for multilingual broadcast and institutional speech.

Findings

Up to 10% reduction in language diarization error

Over 60% reduction in language confusion

More than 8% relative WER reduction in multilingual speech recognition

Abstract

This paper addresses spoken language identification (SLI) and speech recognition of multilingual broadcast and institutional speech, real application scenarios that have been rarely addressed in the SLI literature. Observing that in these domains language changes are mostly associated with speaker changes, we propose a cascaded system consisting of speaker diarization and language identification and compare it with more traditional language identification and language diarization systems. Results show that the proposed system often achieves lower language classification and language diarization error rates (up to 10% relative language diarization error reduction and 60% relative language confusion reduction) and leads to lower WERs on multilingual test sets (more than 8% relative WER reduction), while at the same time does not negatively affect speech recognition on monolingual audio (with an absolute WER increase between 0.1% and 0.7% w.r.t. monolingual ASR).