Smule Renaissance Small: Efficient General-Purpose Vocal Restoration

TL;DR

Smule Renaissance Small (SRS) is a compact, efficient model for end-to-end vocal restoration that handles multiple degradations in real-time, outperforming baselines and matching commercial systems without speech-specific training.

Contribution

The paper introduces SRS, a novel single-stage model for vocal restoration that operates efficiently in real-time and introduces the Extreme Degradation Bench for realistic evaluation.

Findings

SRS outperforms a GAN baseline on DNS 5 Challenge.

SRS matches a flow-matching system's performance.

SRS surpasses open-source baselines on EDB.

Abstract

Vocal recordings on consumer devices commonly suffer from multiple concurrent degradations: noise, reverberation, band-limiting, and clipping. We present Smule Renaissance Small (SRS), a compact single-stage model that performs end-to-end vocal restoration directly in the complex STFT domain. By incorporating phase-aware losses, SRS enables large analysis windows for improved frequency resolution while achieving 10.5x real-time inference on iPhone 12 CPU at 48 kHz. On the DNS 5 Challenge blind set, despite no speech training, SRS outperforms a strong GAN baseline and closely matches a computationally expensive flow-matching system. To enable evaluation under realistic multi-degradation scenarios, we introduce the Extreme Degradation Bench (EDB): 87 singing and speech recordings captured under severe acoustic conditions. On EDB, SRS surpasses all open-source baselines on singing and matches commercial systems, while remaining competitive on speech despite no speech-specific training. We release both SRS and EDB under the MIT License.