MIMII-Agent: Leveraging LLMs with Function Calling for Relative Evaluation of Anomalous Sound Detection

TL;DR

This paper introduces a novel LLM-based method to generate plausible anomalous sounds from normal machine sounds for evaluating unsupervised sound detection systems without needing real anomaly data.

Contribution

It presents a new synthesis approach using LLMs to interpret fault descriptions and select audio transformations, enabling scalable anomaly generation across diverse machine types.

Findings

Synthetic anomalies show consistent detection difficulty trends with real anomalies.

The LLM-based synthesis method effectively evaluates UASD systems.

The approach reduces reliance on real anomalous sound data.

Abstract

This paper proposes a method for generating machine-type-specific anomalies to evaluate the relative performance of unsupervised anomalous sound detection (UASD) systems across different machine types, even in the absence of real anomaly sound data. Conventional keyword-based data augmentation methods often produce unrealistic sounds due to their reliance on manually defined labels, limiting scalability as machine types and anomaly patterns diversify. Advanced audio generative models, such as MIMII-Gen, show promise but typically depend on anomalous training data, making them less effective when diverse anomalous examples are unavailable. To address these limitations, we propose a novel synthesis approach leveraging large language models (LLMs) to interpret textual descriptions of faults and automatically select audio transformation functions, converting normal machine sounds into diverse and plausible anomalous sounds. We validate this approach by evaluating a UASD system trained only on normal sounds from five machine types, using both real and synthetic anomaly data. Experimental results reveal consistent trends in relative detection difficulty across machine types between synthetic and real anomalies. This finding supports our hypothesis and highlights the effectiveness of the proposed LLM-based synthesis approach for relative evaluation of UASD systems.