VRDMG: Vocal Restoration via Diffusion Posterior Sampling with Multiple Guidance

TL;DR

This paper improves diffusion posterior sampling (DPS) for music restoration by addressing performance issues and integrating guidance techniques, leading to superior results in vocal declipping and bandwidth extension tasks.

Contribution

The paper introduces mitigation strategies for DPS limitations using diverse guidance methods, enhancing music restoration performance in challenging scenarios.

Findings

Outperforms existing DPS-based benchmarks in vocal declipping

Achieves better bandwidth extension results under various distortions

Demonstrates versatility across multiple music restoration tasks

Abstract

Restoring degraded music signals is essential to enhance audio quality for downstream music manipulation. Recent diffusion-based music restoration methods have demonstrated impressive performance, and among them, diffusion posterior sampling (DPS) stands out given its intrinsic properties, making it versatile across various restoration tasks. In this paper, we identify that there are potential issues which will degrade current DPS-based methods' performance and introduce the way to mitigate the issues inspired by diverse diffusion guidance techniques including the RePaint (RP) strategy and the Pseudoinverse-Guided Diffusion Models ($\Pi$GDM). We demonstrate our methods for the vocal declipping and bandwidth extension tasks under various levels of distortion and cutoff frequency, respectively. In both tasks, our methods outperform the current DPS-based music restoration benchmarks. We refer to \url{http://carlosholivan.github.io/demos/audio-restoration-2023.html} for examples of the restored audio samples.