Bridge-SR: Schr\"odinger Bridge for Efficient SR

TL;DR

Bridge-SR introduces a novel Schr"odinger Bridge-based approach for efficient speech super-resolution, leveraging low-resolution waveforms as priors to generate high-quality high-resolution speech efficiently and outperforming existing methods in quality and speed.

Contribution

The paper proposes a new Schr"odinger Bridge model for speech super-resolution that uses a lightweight network and data-to-data generation, improving quality and inference speed over prior methods.

Findings

Outperforms baseline methods in speech SR quality on VCTK dataset.

Achieves faster inference with fewer steps compared to diffusion models.

Uses a lightweight network (1.7M parameters) for efficient high-quality SR.

Abstract

Speech super-resolution (SR), which generates a waveform at a higher sampling rate from its low-resolution version, is a long-standing critical task in speech restoration. Previous works have explored speech SR in different data spaces, but these methods either require additional compression networks or exhibit limited synthesis quality and inference speed. Motivated by recent advances in probabilistic generative models, we present Bridge-SR, a novel and efficient any-to-48kHz SR system in the speech waveform domain. Using tractable Schr\"odinger Bridge models, we leverage the observed low-resolution waveform as a prior, which is intrinsically informative for the high-resolution target. By optimizing a lightweight network to learn the score functions from the prior to the target, we achieve efficient waveform SR through a data-to-data generation process that fully exploits the instructive content contained in the low-resolution observation. Furthermore, we identify the importance of the noise schedule, data scaling, and auxiliary loss functions, which further improve the SR quality of bridge-based systems. The experiments conducted on the benchmark dataset VCTK demonstrate the efficiency of our system: (1) in terms of sample quality, Bridge-SR outperforms several strong baseline methods under different SR settings, using a lightweight network backbone (1.7M); (2) in terms of inference speed, our 4-step synthesis achieves better performance than the 8-step conditional diffusion counterpart (LSD: 0.911 vs 0.927). Demo at https://bridge-sr.github.io.