ASDKit: A Toolkit for Comprehensive Evaluation of Anomalous Sound Detection Methods

TL;DR

ASDKit is an open-source toolkit designed to standardize and facilitate the evaluation of diverse anomalous sound detection methods across multiple datasets, promoting reproducibility and comprehensive assessment.

Contribution

It provides a unified framework with training and evaluation scripts for various ASD methods, supporting extensive benchmarking on recent datasets.

Findings

Re-evaluated multiple ASD methods showing consistent effectiveness.

Achieved state-of-the-art performance on DCASE datasets.

Facilitated reproducibility and comprehensive assessment of ASD techniques.

Abstract

In this paper, we introduce ASDKit, a toolkit for anomalous sound detection (ASD) task. Our aim is to facilitate ASD research by providing an open-source framework that collects and carefully evaluates various ASD methods. First, ASDKit provides training and evaluation scripts for a wide range of ASD methods, all handled within a unified framework. For instance, it includes the autoencoder-based official DCASE baseline, representative discriminative methods, and self-supervised learning-based methods. Second, it supports comprehensive evaluation on the DCASE 2020--2024 datasets, enabling careful assessment of ASD performance, which is highly sensitive to factors such as datasets and random seeds. In our experiments, we re-evaluate various ASD methods using ASDKit and identify consistently effective techniques across multiple datasets and trials. We also demonstrate that ASDKit reproduces the state-of-the-art-level performance on the considered datasets.