# Collaborative Deep Learning for Speech Enhancement: A Run-Time Model Selection Method Using Autoencoders

## TL;DR

This paper introduces a collaborative deep learning approach for speech enhancement that dynamically selects the best DNN module during runtime using an autoencoder-based quality assessment, improving performance over fixed models.

## Contribution

It proposes a modular neural network framework with autoencoder-based run-time module selection for speech enhancement, enabling effective reuse of pre-trained models without retraining.

## Key findings

- Almost always outperforms arbitrary module selection
- Sometimes matches oracle-level performance
- Effective for various noise types and SNR levels

## Abstract

We show that a Modular Neural Network (MNN) can combine various speech enhancement modules, each of which is a Deep Neural Network (DNN) specialized on a particular enhancement job. Differently from an ordinary ensemble technique that averages variations in models, the propose MNN selects the best module for the unseen test signal to produce a greedy ensemble. We see this as Collaborative Deep Learning (CDL), because it can reuse various already-trained DNN models without any further refining. In the proposed MNN selecting the best module during run time is challenging. To this end, we employ a speech AutoEncoder (AE) as an arbitrator, whose input and output are trained to be as similar as possible if its input is clean speech. Therefore, the AE can gauge the quality of the module-specific denoised result by seeing its AE reconstruction error, e.g. low error means that the module output is similar to clean speech. We propose an MNN structure with various modules that are specialized on dealing with a specific noise type, gender, and input Signal-to-Noise Ratio (SNR) value, and empirically prove that it almost always works better than an arbitrarily chosen DNN module and sometimes as good as an oracle result.