Full-band General Audio Synthesis with Score-based Diffusion

TL;DR

This paper introduces DAG, a diffusion-based model for full-band general audio synthesis that outperforms existing methods in quality and diversity, operating end-to-end in the waveform domain.

Contribution

The paper presents DAG, a novel diffusion model capable of full-band audio synthesis directly in waveform, surpassing prior band-limited models in quality and diversity.

Findings

DAG outperforms existing label-conditioned generators in quality and diversity.

Full-band DAG achieves up to 65% improvement over state-of-the-art.

DAG is flexible for various conditioning schemas.

Abstract

Recent works have shown the capability of deep generative models to tackle general audio synthesis from a single label, producing a variety of impulsive, tonal, and environmental sounds. Such models operate on band-limited signals and, as a result of an autoregressive approach, they are typically conformed by pre-trained latent encoders and/or several cascaded modules. In this work, we propose a diffusion-based generative model for general audio synthesis, named DAG, which deals with full-band signals end-to-end in the waveform domain. Results show the superiority of DAG over existing label-conditioned generators in terms of both quality and diversity. More specifically, when compared to the state of the art, the band-limited and full-band versions of DAG achieve relative improvements that go up to 40 and 65%, respectively. We believe DAG is flexible enough to accommodate different conditioning schemas while providing good quality synthesis.