HD-DEMUCS: General Speech Restoration with Heterogeneous Decoders

TL;DR

HD-DEMUCS is a neural speech restoration model that uses two specialized decoders working in parallel within a U-Net framework, improving restoration quality across diverse distortion environments.

Contribution

The paper introduces a novel parallel decoding architecture with heterogeneous decoders for comprehensive speech restoration, outperforming traditional cascaded or single-decoder systems.

Findings

Outperforms single-decoder systems in various environments

Effective in diverse distortion scenarios

Leverages learnable weighting for improved results

Abstract

This paper introduces an end-to-end neural speech restoration model, HD-DEMUCS, demonstrating efficacy across multiple distortion environments. Unlike conventional approaches that employ cascading frameworks to remove undesirable noise first and then restore missing signal components, our model performs these tasks in parallel using two heterogeneous decoder networks. Based on the U-Net style encoder-decoder framework, we attach an additional decoder so that each decoder network performs noise suppression or restoration separately. We carefully design each decoder architecture to operate appropriately depending on its objectives. Additionally, we improve performance by leveraging a learnable weighting factor, aggregating the two decoder output waveforms. Experimental results with objective metrics across various environments clearly demonstrate the effectiveness of our approach over a single decoder or multi-stage systems for general speech restoration task.