VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration

TL;DR

VoiceFixer is a novel unified framework that effectively restores high-fidelity speech from multiple simultaneous distortions, significantly improving quality of degraded and historical speech recordings through a two-stage neural approach.

Contribution

It introduces VoiceFixer, the first unified model capable of handling multiple distortions and expanding degraded speech to high-fidelity full-bandwidth audio.

Findings

Effective on severely degraded real-world speech

Outperforms prior single-distortion methods

Achieves high subjective and objective quality scores

Abstract

Speech restoration aims to remove distortions in speech signals. Prior methods mainly focus on a single type of distortion, such as speech denoising or dereverberation. However, speech signals can be degraded by several different distortions simultaneously in the real world. It is thus important to extend speech restoration models to deal with multiple distortions. In this paper, we introduce VoiceFixer, a unified framework for high-fidelity speech restoration. VoiceFixer restores speech from multiple distortions (e.g., noise, reverberation, and clipping) and can expand degraded speech (e.g., noisy speech) with a low bandwidth to 44.1 kHz full-bandwidth high-fidelity speech. We design VoiceFixer based on (1) an analysis stage that predicts intermediate-level features from the degraded speech, and (2) a synthesis stage that generates waveform using a neural vocoder. Both objective and subjective evaluations show that VoiceFixer is effective on severely degraded speech, such as real-world historical speech recordings. Samples of VoiceFixer are available at https://haoheliu.github.io/voicefixer.