DHAuDS: A Dynamic and Heterogeneous Audio Benchmark for Test-Time Adaptation

TL;DR

DHAuDS is a comprehensive benchmark for evaluating test-time adaptation methods in audio classification under realistic, dynamic, and heterogeneous acoustic domain shifts, promoting more robust and generalizable audio models.

Contribution

This paper introduces DHAuDS, a novel benchmark with dynamic and diverse noise conditions for assessing TTA methods in audio classification, addressing limitations of previous fixed-noise evaluations.

Findings

DHAuDS enables fair and reproducible comparison of TTA algorithms.

It includes four benchmarks with dynamic corruption levels and heterogeneous noise types.

The framework defines 14 evaluation criteria per benchmark for comprehensive assessment.

Abstract

Audio classifiers frequently face domain shift, when models trained on one dataset lose accuracy on data recorded in acoustically different conditions. Previous Test-Time Adaptation (TTA) research in speech and sound analysis often evaluates models under fixed or mismatched noise settings, that fail to mimic real-world variability. To overcome these limitations, this paper presents DHAuDS (Dynamic and Heterogeneous Audio Domain Shift), a benchmark designed to assess TTA approaches under more realistic and diverse acoustic shifts. DHAuDS comprises four standardized benchmarks: UrbanSound8K-C, SpeechCommandsV2-C, VocalSound-C, and ReefSet-C, each constructed with dynamic corruption severity levels and heterogeneous noise types to simulate authentic audio degradation scenarios. The framework defines 14 evaluation criteria for each benchmark (8 for UrbanSound8K-C), resulting in 50 unrepeated criteria (124 experiments) that collectively enable fair, reproducible, and cross-domain comparison of TTA algorithms. Through the inclusion of dynamic and mixed-domain noise settings, DHAuDS offers a consistent and publicly reproducible testbed to support ongoing studies in robust and adaptive audio modeling.