Towards Explicit Acoustic Evidence Perception in Audio LLMs for Speech Deepfake Detection

TL;DR

This paper introduces SDD-APALLM, an acoustically enhanced audio LLM framework that explicitly exposes fine-grained acoustic cues, significantly improving speech deepfake detection accuracy and robustness by combining raw audio and spectrograms.

Contribution

The paper presents a novel framework that enhances audio LLMs with explicit acoustic evidence exposure, addressing the bias towards semantic cues in deepfake detection.

Findings

Improved detection accuracy and robustness in speech deepfake detection.

Effective utilization of both semantic and acoustic cues.

Enhanced detection performance especially when semantic cues are misleading.

Abstract

Speech deepfake detection (SDD) focuses on identifying whether a given speech signal is genuine or has been synthetically generated. Existing audio large language model (LLM)-based methods excel in content understanding; however, their predictions are often biased toward semantically correlated cues, which results in fine-grained acoustic artifacts being overlooked during the decisionmaking process. Consequently, fake speech with natural semantics can bypass detectors despite harboring subtle acoustic anomalies; this suggests that the challenge stems not from the absence of acoustic data, but from its inadequate accessibility when semantic-dominant reasoning prevails. To address this issue, we investigate SDD within the audio LLM paradigm and introduce SDD with Auditory Perception-enhanced Audio Large Language Model (SDD-APALLM), an acoustically enhanced framework designed to explicitly expose fine-grained time-frequency evidence as accessible acoustic cues. By combining raw audio with structured spectrograms, the proposed framework empowers audio LLMs to more effectively capture subtle acoustic inconsistencies without compromising their semantic understanding. Experimental results indicate consistent gains in detection accuracy and robustness, especially in cases where semantic cues are misleading. Further analysis reveals that these improvements stem from a coordinated utilization of semantic and acoustic information, as opposed to simple modality aggregation.