VoxEffects: A Speech-Oriented Audio Effects Dataset and Benchmark

TL;DR

VoxEffects is a new speech audio effects dataset and benchmark designed to facilitate systematic study of post-production effects, enabling effect identification and robustness evaluation.

Contribution

It introduces a comprehensive dataset with effect annotations and an evaluation benchmark for speech-oriented audio effects analysis.

Findings

Baseline performance established with AudioMAE-based multi-task model.

Analysis of domain shift, robustness, input duration, and gender fairness.

Dataset supports offline synthesis and real-time rendering for training and evaluation.

Abstract

Speech audio in the wild is often processed by post-production effects, but existing speech datasets rarely provide precise annotations of effects and parameters, limiting systematic study. We introduce VoxEffects, a speech audio effects dataset that pairs produced speech with exact effect-chain supervision at multiple granularities. VoxEffects supports speech-oriented audio effect identification: given a produced waveform, infer which effects are present and how they are applied. Built from minimally edited clean speech, it provides an extensible rendering pipeline for both offline synthesis and on-the-fly rendering for efficient training and evaluation. The audio effect identification benchmark includes effect presence detection, preset classification, and intensity prediction, with a robustness protocol covering capture-side and platform-side degradations. We provide an AudioMAE-based multi-task baseline and analyses of domain shift, robustness, input duration, and gender fairness.