On the Connection of Generative Models and Discriminative Models for Anomaly Detection

TL;DR

This paper explores the limitations of generative models in anomaly detection due to multi-peaked data distributions and proposes integrating discriminative methods with GMMs to improve detection performance.

Contribution

It introduces a new perspective on GM-based AD limitations and proposes DiGMM, a method that combines generative and discriminative approaches for enhanced anomaly detection.

Findings

GMMs struggle with multi-peaked distributions in AD.

Integrating discriminative ideas with GMMs can address these limitations.

The proposed DiGMM framework links generative and discriminative models for AD.

Abstract

Anomaly detection (AD) has attracted considerable attention in both academia and industry. Due to the lack of anomalous data in many practical cases, AD is usually solved by first modeling the normal data pattern and then determining if data fit this model. Generative models (GMs) seem a natural tool to achieve this purpose, which learn the normal data distribution and estimate it using a probability density function (PDF). However, some works have observed the ideal performance of such GM-based AD methods. In this paper, we propose a new perspective on the ideal performance of GM-based AD methods. We state that in these methods, the implicit assumption that connects GMs'results to AD's goal is usually implausible due to normal data's multi-peaked distribution characteristic, which is quite common in practical cases. We first qualitatively formulate this perspective, and then focus on the Gaussian mixture model (GMM) to intuitively illustrate the perspective, which is a typical GM and has the natural property to approximate multi-peaked distributions. Based on the proposed perspective, in order to bypass the implicit assumption in the GMM-based AD method, we suggest integrating the Discriminative idea to orient GMM to AD tasks (DiGMM). With DiGMM, we establish a connection of generative and discriminative models, which are two key paradigms for AD and are usually treated separately before. This connection provides a possible direction for future works to jointly consider the two paradigms and incorporate their complementary characteristics for AD.