SIGTYP 2021 Shared Task: Robust Spoken Language Identification

TL;DR

This paper discusses a shared task focused on developing robust spoken language identification systems capable of handling domain and speaker variability, highlighting current challenges and the need for further research in low-resource scenarios.

Contribution

It introduces a shared task that evaluates language identification systems under domain and speaker mismatch conditions, emphasizing the importance of robustness in low-resource settings.

Findings

Current methods perform above 95% accuracy in-domain

Domain adaptation improves performance but challenges remain

Handling speaker and domain variability is still difficult

Abstract

While language identification is a fundamental speech and language processing task, for many languages and language families it remains a challenging task. For many low-resource and endangered languages this is in part due to resource availability: where larger datasets exist, they may be single-speaker or have different domains than desired application scenarios, demanding a need for domain and speaker-invariant language identification systems. This year's shared task on robust spoken language identification sought to investigate just this scenario: systems were to be trained on largely single-speaker speech from one domain, but evaluated on data in other domains recorded from speakers under different recording circumstances, mimicking realistic low-resource scenarios. We see that domain and speaker mismatch proves very challenging for current methods which can perform above 95% accuracy in-domain, which domain adaptation can address to some degree, but that these conditions merit further investigation to make spoken language identification accessible in many scenarios.