Adaptive re-calibration of channel-wise features for Adversarial Audio Classification

TL;DR

This paper introduces a novel channel-wise feature recalibration method using attention fusion for synthetic speech detection, improving generalization across various generative models and outperforming existing detection techniques.

Contribution

It proposes a new attention-based feature fusion approach with Squeeze Excitation blocks for better synthetic speech detection and demonstrates enhanced feature representations combining LFCC and MFCC.

Findings

Achieved 95% accuracy on FoR dataset

Attentional feature fusion improves detection performance

Combining LFCC and MFCC enhances model generalization

Abstract

DeepFake Audio, unlike DeepFake images and videos, has been relatively less explored from detection perspective, and the solutions which exist for the synthetic speech classification either use complex networks or dont generalize to different varieties of synthetic speech obtained using different generative and optimization-based methods. Through this work, we propose a channel-wise recalibration of features using attention feature fusion for synthetic speech detection and compare its performance against different detection methods including End2End models and Resnet-based models on synthetic speech generated using Text to Speech and Vocoder systems like WaveNet, WaveRNN, Tactotron, and WaveGlow. We also experiment with Squeeze Excitation (SE) blocks in our Resnet models and found that the combination was able to get better performance. In addition to the analysis, we also demonstrate that the combination of Linear frequency cepstral coefficients (LFCC) and Mel Frequency cepstral coefficients (MFCC) using the attentional feature fusion technique creates better input features representations which can help even simpler models generalize well on synthetic speech classification tasks. Our models (Resnet based using feature fusion) trained on Fake or Real (FoR) dataset and were able to achieve 95% test accuracy with the FoR data, and an average of 90% accuracy with samples we generated using different generative models after adapting this framework.