Rehearsal with Auxiliary-Informed Sampling for Audio Deepfake Detection

TL;DR

This paper introduces RAIS, a rehearsal-based continual learning method with auxiliary-informed sampling that significantly improves audio deepfake detection by better capturing audio diversity and reducing forgetting.

Contribution

The paper proposes RAIS, a novel rehearsal-based continual learning approach utilizing auxiliary labels for diverse sample selection in audio deepfake detection.

Findings

RAIS achieves an average EER of 1.953% across five experiments.

RAIS outperforms existing state-of-the-art methods.

Auxiliary-informed sampling enhances diversity and reduces bias in rehearsal.

Abstract

The performance of existing audio deepfake detection frameworks degrades when confronted with new deepfake attacks. Rehearsal-based continual learning (CL), which updates models using a limited set of old data samples, helps preserve prior knowledge while incorporating new information. However, existing rehearsal techniques don't effectively capture the diversity of audio characteristics, introducing bias and increasing the risk of forgetting. To address this challenge, we propose Rehearsal with Auxiliary-Informed Sampling (RAIS), a rehearsal-based CL approach for audio deepfake detection. RAIS employs a label generation network to produce auxiliary labels, guiding diverse sample selection for the memory buffer. Extensive experiments show RAIS outperforms state-of-the-art methods, achieving an average Equal Error Rate (EER) of 1.953 % across five experiences. The code is available at: https://github.com/falihgoz/RAIS.