General Purpose Audio Effect Removal

TL;DR

This paper introduces a new approach for removing multiple audio effects from recordings, using a dataset and a dynamic model composition method called RemFX, which outperforms single-effect models.

Contribution

The paper presents the first general-purpose audio effect removal framework that handles multiple effects and sources, using a novel dynamic model composition approach.

Findings

RemFX outperforms single-effect models in effect removal tasks.

A new dataset with five effects across four sources was created for training and evaluation.

Handling multiple effects remains challenging with many effects present.

Abstract

Although the design and application of audio effects is well understood, the inverse problem of removing these effects is significantly more challenging and far less studied. Recently, deep learning has been applied to audio effect removal; however, existing approaches have focused on narrow formulations considering only one effect or source type at a time. In realistic scenarios, multiple effects are applied with varying source content. This motivates a more general task, which we refer to as general purpose audio effect removal. We developed a dataset for this task using five audio effects across four different sources and used it to train and evaluate a set of existing architectures. We found that no single model performed optimally on all effect types and sources. To address this, we introduced RemFX, an approach designed to mirror the compositionality of applied effects. We first trained a set of the best-performing effect-specific removal models and then leveraged an audio effect classification model to dynamically construct a graph of our models at inference. We found our approach to outperform single model baselines, although examples with many effects present remain challenging.