Dynamic Frequency-Adaptive Knowledge Distillation for Speech Enhancement

TL;DR

This paper proposes a dynamic frequency-adaptive knowledge distillation method that improves speech enhancement model compression by tailoring learning objectives to different frequency components, leading to better performance on resource-limited devices.

Contribution

The novel DFKD approach dynamically adapts distillation objectives based on frequency content, enhancing speech enhancement model compression and performance.

Findings

Significant performance improvements in compressed models.

Outperforms existing logit-based distillation methods.

Effective across multiple state-of-the-art models.

Abstract

Deep learning-based speech enhancement (SE) models have recently outperformed traditional techniques, yet their deployment on resource-constrained devices remains challenging due to high computational and memory demands. This paper introduces a novel dynamic frequency-adaptive knowledge distillation (DFKD) approach to effectively compress SE models. Our method dynamically assesses the model's output, distinguishing between high and low-frequency components, and adapts the learning objectives to meet the unique requirements of different frequency bands, capitalizing on the SE task's inherent characteristics. To evaluate the DFKD's efficacy, we conducted experiments on three state-of-the-art models: DCCRN, ConTasNet, and DPTNet. The results demonstrate that our method not only significantly enhances the performance of the compressed model (student model) but also surpasses other logit-based knowledge distillation methods specifically for SE tasks.