Retrieval-Augmented Approach for Unsupervised Anomalous Sound Detection and Captioning without Model Training

TL;DR

This paper introduces a retrieval-augmented, unsupervised method for anomalous sound detection and captioning that leverages pre-trained models, eliminating the need for training data and ensuring caption relevance.

Contribution

It presents a novel retrieval-based approach for captioning anomalous sounds in an unsupervised manner using pre-trained models, avoiding separate training and irrelevant captions.

Findings

Effective anomalous sound detection without training

Captions are consistent with detection results

Subjective evaluation confirms method's effectiveness

Abstract

This paper proposes a method for unsupervised anomalous sound detection (UASD) and captioning the reason for detection. While there is a method that captions the difference between given normal and anomalous sound pairs, it is assumed to be trained and used separately from the UASD model. Therefore, the obtained caption can be irrelevant to the differences that the UASD model captured. In addition, it requires many caption labels representing differences between anomalous and normal sounds for model training. The proposed method employs a retrieval-augmented approach for captioning of anomalous sounds. Difference captioning in the embedding space output by the pre-trained CLAP (contrastive language-audio pre-training) model makes the anomalous sound detection results consistent with the captions and does not require training. Experiments based on subjective evaluation and a sample-wise analysis of the output captions demonstrate the effectiveness of the proposed method.