Differentiable Modelling of Percussive Audio with Transient and Spectral Synthesis

TL;DR

This paper introduces a differentiable synthesis framework for percussive sounds that explicitly models transients using sinusoidal and transient encoders, improving the reconstruction of drum sounds.

Contribution

It presents a novel DDSP-based model combining sinusoidal modeling with transient generation via temporal convolutional networks for percussive audio synthesis.

Findings

Improved onset signal reconstruction for membranophone percussion.

Effective joint training of noise, transient, and sinusoidal encoders.

Enhanced interpretability of percussive sound synthesis.

Abstract

Differentiable digital signal processing (DDSP) techniques, including methods for audio synthesis, have gained attention in recent years and lend themselves to interpretability in the parameter space. However, current differentiable synthesis methods have not explicitly sought to model the transient portion of signals, which is important for percussive sounds. In this work, we present a unified synthesis framework aiming to address transient generation and percussive synthesis within a DDSP framework. To this end, we propose a model for percussive synthesis that builds on sinusoidal modeling synthesis and incorporates a modulated temporal convolutional network for transient generation. We use a modified sinusoidal peak picking algorithm to generate time-varying non-harmonic sinusoids and pair it with differentiable noise and transient encoders that are jointly trained to reconstruct drumset sounds. We compute a set of reconstruction metrics using a large dataset of acoustic and electronic percussion samples that show that our method leads to improved onset signal reconstruction for membranophone percussion instruments.