Phoneme-Level Analysis for Person-of-Interest Speech Deepfake Detection

TL;DR

This paper introduces a phoneme-level POI-based speech deepfake detection method that enhances interpretability and robustness by analyzing individual phonemes to identify synthetic speech artifacts.

Contribution

It presents a novel phoneme-level approach for POI-based deepfake detection, improving interpretability and robustness over existing methods.

Findings

Achieves comparable accuracy to traditional methods.

Offers superior robustness and interpretability.

Enables fine-grained detection of synthetic artifacts.

Abstract

Recent advances in generative AI have made the creation of speech deepfakes widely accessible, posing serious challenges to digital trust. To counter this, various speech deepfake detection strategies have been proposed, including Person-of-Interest (POI) approaches, which focus on identifying impersonations of specific individuals by modeling and analyzing their unique vocal traits. Despite their excellent performance, the existing methods offer limited granularity and lack interpretability. In this work, we propose a POI-based speech deepfake detection method that operates at the phoneme level. Our approach decomposes reference audio into phonemes to construct a detailed speaker profile. In inference, phonemes from a test sample are individually compared against this profile, enabling fine-grained detection of synthetic artifacts. The proposed method achieves comparable accuracy to traditional approaches while offering superior robustness and interpretability, key aspects in multimedia forensics. By focusing on phoneme analysis, this work explores a novel direction for explainable, speaker-centric deepfake detection.