Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0

TL;DR

This paper introduces a Mixture of Experts fusion method that leverages frozen pretrained wav2vec 2.0 features for efficient fake audio detection, avoiding fine-tuning and maintaining high performance.

Contribution

The proposed method enables effective feature fusion for fake audio detection using frozen pretrained models, reducing training time and facilitating rapid adaptation to new speech synthesis techniques.

Findings

Achieves competitive detection performance on ASVspoof datasets

Reduces training time by avoiding fine-tuning of pretrained models

Effectively integrates multi-layer features guided by a gating network

Abstract

Speech synthesis technology has posed a serious threat to speaker verification systems. Currently, the most effective fake audio detection methods utilize pretrained models, and integrating features from various layers of pretrained model further enhances detection performance. However, most of the previously proposed fusion methods require fine-tuning the pretrained models, resulting in excessively long training times and hindering model iteration when facing new speech synthesis technology. To address this issue, this paper proposes a feature fusion method based on the Mixture of Experts, which extracts and integrates features relevant to fake audio detection from layer features, guided by a gating network based on the last layer feature, while freezing the pretrained model. Experiments conducted on the ASVspoof2019 and ASVspoof2021 datasets demonstrate that the proposed method achieves competitive performance compared to those requiring fine-tuning.