LASPA: Language Agnostic Speaker Disentanglement with Prefix-Tuned Cross-Attention

TL;DR

This paper introduces LASPA, a novel language-agnostic speaker disentanglement method using prefix-tuned cross-attention, which improves multi-lingual speaker recognition by effectively separating linguistic and speaker information.

Contribution

The paper presents a new disentanglement learning strategy with prefix-tuned cross-attention that enhances speaker recognition across multiple languages, including unseen ones.

Findings

Improves equal error rate across multiple datasets

Effectively separates language from speaker embeddings

Generalizes well to unseen languages

Abstract

Speaker recognition models face challenges in multi-lingual settings due to the entanglement of linguistic information within speaker embeddings. The overlap between vocal traits such as accent, vocal anatomy, and a language's phonetic structure complicates separating linguistic and speaker information. Disentangling these components can significantly improve speaker recognition accuracy. To this end, we propose a novel disentanglement learning strategy that integrates joint learning through prefix-tuned cross-attention. This approach is particularly effective when speakers switch between languages. Experimental results show the model generalizes across monolingual and multi-lingual settings, including unseen languages. Notably, the proposed model improves the equal error rate across multiple datasets, highlighting its ability to separate language information from speaker embeddings and enhance recognition in diverse linguistic conditions.