Lightweight Model Attribution and Detection of Synthetic Speech via Audio Residual Fingerprints

TL;DR

This paper introduces a lightweight, training-free method for detecting and attributing synthetic speech to its source model using residual fingerprints, achieving high accuracy and robustness across various conditions.

Contribution

The paper presents a novel residual fingerprint approach for synthetic speech detection and attribution that is simple, model-agnostic, and effective without training.

Findings

AUROC scores above 99% across multiple systems

High robustness to audio distortions like echo and noise

Effective out-of-domain detection with F1 score of 0.91

Abstract

As speech generation technologies advance, so do risks of impersonation, misinformation, and spoofing. We present a lightweight, training-free approach for detecting synthetic speech and attributing it to its source model. Our method addresses three tasks: (1) single-model attribution in an open-world setting, (2) multi-model attribution in a closed-world setting, and (3) real vs. synthetic speech classification. The core idea is simple: we compute standardized average residuals--the difference between an audio signal and its filtered version--to extract model-agnostic fingerprints that capture synthesis artifacts. Experiments across multiple synthesis systems and languages show AUROC scores above 99%, with strong reliability even when only a subset of model outputs is available. The method maintains high performance under common audio distortions, including echo and moderate background noise, while data augmentation can improve results in more challenging conditions. In addition, out-of-domain detection is performed using Mahalanobis distances to in-domain residual fingerprints, achieving an F1 score of 0.91 on unseen models, reinforcing the method's efficiency, generalizability, and suitability for digital forensics and security applications.