Modulation Extraction for LFO-driven Audio Effects

TL;DR

This paper introduces a method to extract LFO signals from processed audio, enabling modeling of effects without needing internal effect parameters, and provides a real-time plugin for practical use.

Contribution

It presents a novel framework for extracting arbitrary LFO signals from audio, facilitating end-to-end modeling of LFO-driven effects without internal effect access.

Findings

Effective extraction of quasiperiodic and distorted LFO signals

Enables training of black-box effect models using only dry and wet audio

Provides a real-time VST plugin for practical application

Abstract

Low frequency oscillator (LFO) driven audio effects such as phaser, flanger, and chorus, modify an input signal using time-varying filters and delays, resulting in characteristic sweeping or widening effects. It has been shown that these effects can be modeled using neural networks when conditioned with the ground truth LFO signal. However, in most cases, the LFO signal is not accessible and measurement from the audio signal is nontrivial, hindering the modeling process. To address this, we propose a framework capable of extracting arbitrary LFO signals from processed audio across multiple digital audio effects, parameter settings, and instrument configurations. Since our system imposes no restrictions on the LFO signal shape, we demonstrate its ability to extract quasiperiodic, combined, and distorted modulation signals that are relevant to effect modeling. Furthermore, we show how coupling the extraction model with a simple processing network enables training of end-to-end black-box models of unseen analog or digital LFO-driven audio effects using only dry and wet audio pairs, overcoming the need to access the audio effect or internal LFO signal. We make our code available and provide the trained audio effect models in a real-time VST plugin.