Neighbors and relatives: How do speech embeddings reflect linguistic connections across the world?

TL;DR

This paper explores how speech embeddings derived from machine learning models can reveal linguistic relationships across the world, aligning well with traditional measures and offering scalable, data-driven insights into language variation.

Contribution

It demonstrates the effectiveness of speech embeddings from a self-supervised model in capturing linguistic relationships, enabling large-scale analysis of language connections beyond traditional methods.

Findings

Embedding-based distances align with genealogical, lexical, and geographical measures.

Speech embeddings effectively capture global and local linguistic patterns.

Method shows promise for analyzing low-resource languages and linguistic diversity.

Abstract

Investigating linguistic relationships on a global scale requires analyzing diverse features such as syntax, phonology and prosody, which evolve at varying rates influenced by internal diversification, language contact, and sociolinguistic factors. Recent advances in machine learning (ML) offer complementary alternatives to traditional historical and typological approaches. Instead of relying on expert labor in analyzing specific linguistic features, these new methods enable the exploration of linguistic variation through embeddings derived directly from speech, opening new avenues for large-scale, data-driven analyses. This study employs embeddings from the fine-tuned XLS-R self-supervised language identification model voxlingua107-xls-r-300m-wav2vec, to analyze relationships between 106 world languages based on speech recordings. Using linear discriminant analysis (LDA), language embeddings are clustered and compared with genealogical, lexical, and geographical distances. The results demonstrate that embedding-based distances align closely with traditional measures, effectively capturing both global and local typological patterns. Challenges in visualizing relationships, particularly with hierarchical clustering and network-based methods, highlight the dynamic nature of language change. The findings show potential for scalable analyses of language variation based on speech embeddings, providing new perspectives on relationships among languages. By addressing methodological considerations such as corpus size and latent space dimensionality, this approach opens avenues for studying low-resource languages and bridging macro- and micro-level linguistic variation. Future work aims to extend these methods to underrepresented languages and integrate sociolinguistic variation for a more comprehensive understanding of linguistic diversity.