NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling

TL;DR

NU-Wave is a novel diffusion probabilistic model that significantly improves neural audio upsampling from 16kHz/24kHz to 48kHz, outperforming baselines with smaller model size.

Contribution

It introduces NU-Wave, the first diffusion-based neural audio super-resolution model capable of high-quality upsampling to 48kHz.

Findings

Outperforms baseline models in SNR, LSD, and ABX accuracy.

Achieves high-quality audio with fewer parameters.

First diffusion probabilistic model for audio super-resolution.

Abstract

In this work, we introduce NU-Wave, the first neural audio upsampling model to produce waveforms of sampling rate 48kHz from coarse 16kHz or 24kHz inputs, while prior works could generate only up to 16kHz. NU-Wave is the first diffusion probabilistic model for audio super-resolution which is engineered based on neural vocoders. NU-Wave generates high-quality audio that achieves high performance in terms of signal-to-noise ratio (SNR), log-spectral distance (LSD), and accuracy of the ABX test. In all cases, NU-Wave outperforms the baseline models despite the substantially smaller model capacity (3.0M parameters) than baselines (5.4-21%). The audio samples of our model are available at https://mindslab-ai.github.io/nuwave, and the code will be made available soon.