Disentangling Speaker Traits for Deepfake Source Verification via Chebyshev Polynomial and Riemannian Metric Learning

TL;DR

This paper introduces a novel metric learning framework that disentangles speaker traits from source embeddings in deepfake speech verification, utilizing Chebyshev polynomials and Riemannian geometry to improve source discrimination.

Contribution

The paper proposes a new SDML framework with two innovative loss functions that enhance speaker disentanglement and source verification accuracy in deepfake speech detection.

Findings

Effective source verification on MLAAD benchmark

Improved disentanglement of speaker traits

Robust performance under new protocols

Abstract

Speech deepfake source verification systems aims to determine whether two synthetic speech utterances originate from the same source generator, often assuming that the resulting source embeddings are independent of speaker traits. However, this assumption remains unverified. In this paper, we first investigate the impact of speaker factors on source verification. We propose a speaker-disentangled metric learning (SDML) framework incorporating two novel loss functions. The first leverages Chebyshev polynomial to mitigate gradient instability during disentanglement optimization. The second projects source and speaker embeddings into hyperbolic space, leveraging Riemannian metric distances to reduce speaker information and learn more discriminative source features. Experimental results on MLAAD benchmark, evaluated under four newly proposed protocols designed for source-speaker disentanglement scenarios, demonstrate the effectiveness of SDML framework. The code, evaluation protocols and demo website are available at https://github.com/xxuan-acoustics/RiemannSD-Net.