HM-Conformer: A Conformer-based audio deepfake detection system with hierarchical pooling and multi-level classification token aggregation methods

TL;DR

HM-Conformer introduces hierarchical pooling and multi-level classification token aggregation to improve audio deepfake detection, effectively capturing local and global spoofing evidence with competitive results on the ASVspoof 2021 dataset.

Contribution

The paper proposes HM-Conformer, a novel Conformer-based system with hierarchical pooling and multi-level token aggregation for enhanced audio deepfake detection.

Findings

Achieved 15.71% EER on ASVspoof 2021 dataset.

Outperforms several recent deepfake detection systems.

Effectively captures local and global spoofing cues.

Abstract

Audio deepfake detection (ADD) is the task of detecting spoofing attacks generated by text-to-speech or voice conversion systems. Spoofing evidence, which helps to distinguish between spoofed and bona-fide utterances, might exist either locally or globally in the input features. To capture these, the Conformer, which consists of Transformers and CNN, possesses a suitable structure. However, since the Conformer was designed for sequence-to-sequence tasks, its direct application to ADD tasks may be sub-optimal. To tackle this limitation, we propose HM-Conformer by adopting two components: (1) Hierarchical pooling method progressively reducing the sequence length to eliminate duplicated information (2) Multi-level classification token aggregation method utilizing classification tokens to gather information from different blocks. Owing to these components, HM-Conformer can efficiently detect spoofing evidence by processing various sequence lengths and aggregating them. In experimental results on the ASVspoof 2021 Deepfake dataset, HM-Conformer achieved a 15.71% EER, showing competitive performance compared to recent systems.