DDSP-SFX: Acoustically-guided sound effects generation with differentiable digital signal processing

TL;DR

DDSP-SFX is a novel neural audio synthesis model that produces high-quality sound effects with controllable timbre and acoustic attributes, leveraging differentiable digital signal processing for deterministic sound manipulation.

Contribution

It introduces DDSP-SFX, a new model that enhances sound effect synthesis with improved transient modeling and controllable timbre variation using a simple, deterministic approach.

Findings

Higher evaluation scores for impulsive sounds

Effective timbre transfer demonstrated

Enhanced control over sound attributes

Abstract

Controlling the variations of sound effects using neural audio synthesis models has been a difficult task. Differentiable digital signal processing (DDSP) provides a lightweight solution that achieves high-quality sound synthesis while enabling deterministic acoustic attribute control by incorporating pre-processed audio features and digital synthesizers. In this research, we introduce DDSP-SFX, a model based on the DDSP architecture capable of synthesizing high-quality sound effects while enabling users to control the timbre variations easily. We propose a transient modelling technique with higher objective evaluation scores and subjective ratings over impulsive signals (footsteps, gunshots). We propose a simple method that achieves timbre variation control while also allowing deterministic attribute control. We further qualitatively show the timbre transfer performance using voice as the guiding sound.