Emotion and Acoustics Should Agree: Cross-Level Inconsistency Analysis for Audio Deepfake Detection

TL;DR

This paper introduces EAI-ADD, a novel audio deepfake detection method that leverages cross-level inconsistencies between emotional and acoustic features to improve spoof speech detection accuracy.

Contribution

The paper proposes a new approach that models cross-level emotion-acoustic inconsistencies, addressing limitations of previous correlation-based methods in audio deepfake detection.

Findings

EAI-ADD outperforms baseline methods on ASVspoof datasets.

Cross-level emotion-acoustic inconsistency is an effective detection signal.

The method captures subtle desynchronizations missed by prior approaches.

Abstract

Audio Deepfake Detection (ADD) aims to detect spoof speech from bonafide speech. Most prior studies assume that stronger correlations within or across acoustic and emotional features imply authenticity, and thus focus on enhancing or measuring such correlations. However, existing methods often treat acoustic and emotional features in isolation or rely on correlation metrics, which overlook subtle desynchronization between them and smooth out abrupt discontinuities. To address these issues, we propose EAI-ADD, which treats cross level emotion acoustic inconsistency as the primary detection signal. We first project emotional and acoustic representations into a comparable space. Then we progressively integrate frame level and utterance level emotion features with acoustic features to capture cross level emotion acoustic inconsistencies across different temporal granularities. Experimental results on the ASVspoof 2019LA and 2021LA datasets demonstrate that the proposed EAI-ADD outperforms baselines, providing a more effective solution for audio anti spoofing detection.