Towards generalisable and calibrated synthetic speech detection with self-supervised representations

TL;DR

This paper demonstrates that using large frozen self-supervised audio representations with simple classifiers significantly improves the generalisation and calibration of deepfake speech detection models, outperforming previous methods.

Contribution

The study shows that pretrained self-supervised representations combined with logistic regression enhance generalisation and calibration in deepfake detection, with fewer parameters and better reliability.

Findings

Reduced EER from 30.9% to 8.8% on eight datasets

More reliable and calibrated predictions compared to previous methods

Achieved strong generalisation with less than 2k parameters

Abstract

Generalisation -- the ability of a model to perform well on unseen data -- is crucial for building reliable deepfake detectors. However, recent studies have shown that the current audio deepfake models fall short of this desideratum. In this work we investigate the potential of pretrained self-supervised representations in building general and calibrated audio deepfake detection models. We show that large frozen representations coupled with a simple logistic regression classifier are extremely effective in achieving strong generalisation capabilities: compared to the RawNet2 model, this approach reduces the equal error rate from 30.9% to 8.8% on a benchmark of eight deepfake datasets, while learning less than 2k parameters. Moreover, the proposed method produces considerably more reliable predictions compared to previous approaches making it more suitable for realistic use.