Exploring Quality and Generalizability in Parameterized Neural Audio Effects

TL;DR

This paper advances neural audio effects modeling by improving architecture and training strategies to enhance quality, efficiency, and generalizability, especially for specific instrument datasets, aiming for professional-grade audio processing.

Contribution

It introduces methods to improve neural network modeling of nonlinear audio effects, focusing on dataset manipulation and architectural changes for better accuracy and efficiency.

Findings

Dataset manipulation with instrument-specific data improves accuracy.

Architectural and optimization changes enhance computational efficiency.

Model generalizability extends to a larger variety of nonlinear effects.

Abstract

Deep neural networks have shown promise for music audio signal processing applications, often surpassing prior approaches, particularly as end-to-end models in the waveform domain. Yet results to date have tended to be constrained by low sample rates, noise, narrow domains of signal types, and/or lack of parameterized controls (i.e. "knobs"), making their suitability for professional audio engineering workflows still lacking. This work expands on prior research published on modeling nonlinear time-dependent signal processing effects associated with music production by means of a deep neural network, one which includes the ability to emulate the parameterized settings you would see on an analog piece of equipment, with the goal of eventually producing commercially viable, high quality audio, i.e. 44.1 kHz sampling rate at 16-bit resolution. The results in this paper highlight progress in modeling these effects through architecture and optimization changes, towards increasing computational efficiency, lowering signal-to-noise ratio, and extending to a larger variety of nonlinear audio effects. Toward these ends, the strategies employed involved a three-pronged approach: model speed, model accuracy, and model generalizability. Most of the presented methods provide marginal or no increase in output accuracy over the original model, with the exception of dataset manipulation. We found that limiting the audio content of the dataset, for example using datasets of just a single instrument, provided a significant improvement in model accuracy over models trained on more general datasets.