StyleWaveGAN: Style-based synthesis of drum sounds with extensive controls using generative adversarial networks

TL;DR

StyleWaveGAN is a style-based generative adversarial network that synthesizes high-quality drum sounds with extensive control, faster-than-real-time performance, and improved quality over existing methods, using conditioning on drum type and audio descriptors.

Contribution

Introducing StyleWaveGAN, a novel style-based drum sound generator that enables extensive control, faster synthesis, and improved quality, along with an alternative to progressive GAN training and dataset balancing insights.

Findings

Achieved real-time capable synthesis of drum sounds on GPU.

Demonstrated superior quality over WaveGAN and NeuroDrum using Frechet Audio Distance.

Showed effective control over drum type and audio descriptors during synthesis.

Abstract

In this paper we introduce StyleWaveGAN, a style-based drum sound generator that is a variation of StyleGAN, a state-of-the-art image generator. By conditioning StyleWaveGAN on both the type of drum and several audio descriptors, we are able to synthesize waveforms faster than real-time on a GPU directly in CD quality up to a duration of 1.5s while retaining a considerable amount of control over the generation. We also introduce an alternative to the progressive growing of GANs and experimented on the effect of dataset balancing for generative tasks. The experiments are carried out on an augmented subset of a publicly available dataset comprised of different drums and cymbals. We evaluate against two recent drum generators, WaveGAN and NeuroDrum, demonstrating significantly improved generation quality (measured with the Frechet Audio Distance) and interesting results with perceptual features.