Modelling black-box audio effects with time-varying feature modulation

TL;DR

This paper introduces a novel method using time-varying feature modulation within neural networks to better model long-range dependencies in black-box audio effects like fuzz and compression, surpassing traditional scaling methods.

Contribution

The paper proposes integrating time-varying feature-wise linear modulation into convolutional architectures to improve long-term dependency modeling in audio effects.

Findings

Enhanced accuracy in modeling fuzz and compressor effects

Better capture of long-range dependencies in audio signals

Provides reproducible code and pretrained models

Abstract

Deep learning approaches for black-box modelling of audio effects have shown promise, however, the majority of existing work focuses on nonlinear effects with behaviour on relatively short time-scales, such as guitar amplifiers and distortion. While recurrent and convolutional architectures can theoretically be extended to capture behaviour at longer time scales, we show that simply scaling the width, depth, or dilation factor of existing architectures does not result in satisfactory performance when modelling audio effects such as fuzz and dynamic range compression. To address this, we propose the integration of time-varying feature-wise linear modulation into existing temporal convolutional backbones, an approach that enables learnable adaptation of the intermediate activations. We demonstrate that our approach more accurately captures long-range dependencies for a range of fuzz and compressor implementations across both time and frequency domain metrics. We provide sound examples, source code, and pretrained models to faciliate reproducibility.