Quantifying Cross-Lingual Transfer in Paralinguistic Speech Tasks

TL;DR

This paper introduces the Cross-Lingual Transfer Matrix (CLTM) to systematically quantify how different languages influence paralinguistic speech tasks like gender identification and speaker verification, revealing language-dependent transfer patterns.

Contribution

The paper presents the CLTM method for systematic analysis of cross-lingual transfer effects in paralinguistic speech tasks, addressing limitations of prior pairwise studies.

Findings

Distinct transfer patterns across tasks and languages

Language-dependent effects in cross-lingual transfer

Systematic analysis of donor-language influence

Abstract

Paralinguistic speech tasks are often considered relatively language-agnostic, as they rely on extralinguistic acoustic cues rather than lexical content. However, prior studies report performance degradation under cross-lingual conditions, indicating non-negligible language dependence. Still, these studies typically focus on isolated language pairs or task-specific settings, limiting comparability and preventing a systematic assessment of task-level language dependence. We introduce the Cross-Lingual Transfer Matrix (CLTM), a systematic method to quantify cross-lingual interactions between pairs of languages within a given task. We apply the CLTM to two paralinguistic tasks, gender identification and speaker verification, using a multilingual HuBERT-based encoder, to analyze how donor-language data affects target-language performance during fine-tuning. Our results reveal distinct transfer patterns across tasks and languages, reflecting systematic, language-dependent effects.