Inference-time Scaling for Diffusion-based Audio Super-resolution

TL;DR

This paper introduces an inference-time scaling method for diffusion-based audio super-resolution that explores multiple solution trajectories to improve output quality without increasing sampling steps, validated across various audio domains.

Contribution

It proposes a novel inference-time scaling paradigm with verifier-guided search algorithms to enhance diffusion model outputs for audio super-resolution.

Findings

Achieved up to 9.70% improvement in aesthetics

Improved speaker similarity by 5.88%

Reduced spectral distance by 46.98% in speech SR

Abstract

Diffusion models have demonstrated remarkable success in generative tasks, including audio super-resolution (SR). In many applications like movie post-production and album mastering, substantial computational budgets are available for achieving superior audio quality. However, while existing diffusion approaches typically increase sampling steps to improve quality, the performance remains fundamentally limited by the stochastic nature of the sampling process, leading to high-variance and quality-limited outputs. Here, rather than simply increasing the number of sampling steps, we propose a different paradigm through inference-time scaling for SR, which explores multiple solution trajectories during the sampling process. Different task-specific verifiers are developed, and two search algorithms, including the random search and zero-order search for SR, are introduced. By actively guiding the exploration of the high-dimensional solution space through verifier-algorithm combinations, we enable more robust and higher-quality outputs. Through extensive validation across diverse audio domains (speech, music, sound effects) and frequency ranges, we demonstrate consistent performance gains, achieving improvements of up to 9.70% in aesthetics, 5.88% in speaker similarity, 15.20% in word error rate, and 46.98% in spectral distance for speech SR from 4kHz to 24kHz, showcasing the effectiveness of our approach. Audio samples are available at: https://racerk.github.io/tt-scale-audiosr/.