TLDiffGAN: A Latent Diffusion-GAN Framework with Temporal Information Fusion for Anomalous Sound Detection

TL;DR

TLDiffGAN introduces a novel framework combining latent diffusion and GANs with temporal feature fusion and augmentation, significantly improving unsupervised anomalous sound detection by capturing complex normal sound features.

Contribution

The paper presents a new diffusion-GAN hybrid framework with temporal information fusion and spectrogram augmentation for enhanced anomalous sound detection.

Findings

Outperforms existing methods on DCASE 2020 dataset

Demonstrates strong localization of anomalies in time-frequency domain

Effectively captures complex normal sound features

Abstract

Existing generative models for unsupervised anomalous sound detection are limited by their inability to fully capture the complex feature distribution of normal sounds, while the potential of powerful diffusion models in this domain remains largely unexplored. To address this challenge, we propose a novel framework, TLDiffGAN, which consists of two complementary branches. One branch incorporates a latent diffusion model into the GAN generator for adversarial training, thereby making the discriminator's task more challenging and improving the quality of generated samples. The other branch leverages pretrained audio model encoders to extract features directly from raw audio waveforms for auxiliary discrimination. This framework effectively captures feature representations of normal sounds from both raw audio and Mel spectrograms. Moreover, we introduce a TMixup spectrogram augmentation technique to enhance sensitivity to subtle and localized temporal patterns that are often overlooked. Extensive experiments on the DCASE 2020 Challenge Task 2 dataset demonstrate the superior detection performance of TLDiffGAN, as well as its strong capability in anomalous time-frequency localization.