adVAE: A self-adversarial variational autoencoder with Gaussian anomaly prior knowledge for anomaly detection

TL;DR

adVAE introduces a self-adversarial training framework for variational autoencoders, leveraging Gaussian anomaly priors to improve unsupervised anomaly detection and prevent overfitting, outperforming state-of-the-art methods.

Contribution

The paper proposes a novel self-adversarial variational autoencoder with Gaussian anomaly priors, enhancing anomaly detection by discriminating normal and anomalous latent variables.

Findings

Significant performance improvements over SOTA baselines.

Effective regularization reduces overfitting in anomaly detection.

Model demonstrates robustness across multiple datasets.

Abstract

Recently, deep generative models have become increasingly popular in unsupervised anomaly detection. However, deep generative models aim at recovering the data distribution rather than detecting anomalies. Besides, deep generative models have the risk of overfitting training samples, which has disastrous effects on anomaly detection performance. To solve the above two problems, we propose a Self-adversarial Variational Autoencoder with a Gaussian anomaly prior assumption. We assume that both the anomalous and the normal prior distribution are Gaussian and have overlaps in the latent space. Therefore, a Gaussian transformer net T is trained to synthesize anomalous but near-normal latent variables. Keeping the original training objective of Variational Autoencoder, besides, the generator G tries to distinguish between the normal latent variables and the anomalous ones synthesized by T, and the encoder E is trained to discriminate whether the output of G is real. These new objectives we added not only give both G and E the ability to discriminate but also introduce additional regularization to prevent overfitting. Compared with the SOTA baselines, the proposed model achieves significant improvements in extensive experiments. Datasets and our model are available at a Github repository.

Tables

Table 1. Table 1: Summary of datasets used in experiments.

Datasets	Size	Dim.	Outliers (percentage)
Letter [40]	1600	32	100 (6.2%)
Cardio [41]	1831	21	176 (9.6%)
Satellite [42]	5100	36	75 (1.5%)
Optical [43]	5216	64	150 (2.8%)
Pen [44]	6870	16	156 (2.3%)

Table 2. Table 2: Summary of hyperparameters used in each dataset.

Datasets	$\lambda$	$m_z$	$m_x$	$\gamma$
Letter [40]	0.003	40	2	0.001
Cardio [41]	0.1	20	2	0.001
Satellite [42]	0.01	40	2	0.001
Optical [43]	0.03	40	2	0.001
Pen [44]	0.01	20	2	0.001

Table 3. Table 3: AP and AUC comparisons between the baselines and adVAE. AP is a better metric than AUC in anomaly detection. Note that the best result is typeset in bold and the second best in italic typeface. The dataset names are represented as their first three letters and Avg. represents the average.

Methods	Average Precision (AP)						Area Under the ROC Curve (AUC)
	Let	Car	Sat	Opt	Pen	Avg.	Let	Car	Sat	Opt	Pen	Avg.
OCSVM [31]	0.306	0.947	0.718	0.157	0.645	0.555	0.514	0.975	0.893	0.613	0.936	0.786
IForest [46]	0.330	0.922	0.784	0.353	0.798	0.637	0.624	0.949	0.945	0.859	0.978	0.871
ABOD [47]	0.769	0.922	0.845	0.762	0.917	0.843	0.915	0.948	0.972	0.969	0.994	0.960
HBOS [49]	0.312	0.851	0.746	0.463	0.641	0.603	0.619	0.899	0.915	0.863	0.942	0.848
SOD [48]	0.542	0.693	0.494	0.105	0.115	0.390	0.792	0.764	0.915	0.405	0.473	0.670
GAN [12]	0.421	0.697	0.427	0.417	0.362	0.465	0.653	0.618	0.776	0.743	0.779	0.714
MOGAAL [24]	0.428	0.730	0.797	0.588	0.308	0.570	0.714	0.792	0.971	0.912	0.838	0.845
WGAN-GP [36]	0.497	0.834	0.765	0.596	0.589	0.656	0.722	0.847	0.920	0.895	0.951	0.867
AE [13]	0.693	0.798	0.764	0.860	0.682	0.759	0.897	0.840	0.950	0.990	0.962	0.928
DAGMM [38]	0.318	0.789	0.353	0.125	0.370	0.391	0.564	0.847	0.714	0.465	0.826	0.683
VAE [13]	0.724	0.803	0.766	0.875	0.625	0.759	0.911	0.840	0.962	0.992	0.959	0.933
G-adVAE	0.723	0.922	0.748	0.885	0.853	0.826	0.908	0.956	0.957	0.993	0.988	0.960
E-adVAE	0.769	0.933	0.801	0.857	0.926	0.857	0.911	0.961	0.968	0.989	0.996	0.965
adVAE	0.779	0.951	0.792	0.957	0.880	0.872	0.921	0.966	0.970	0.996	0.993	0.969

Table 4. Table 4: Recall and F1 score comparisons between the baselines and adVAE. For all reconstruction-based methods (AE, VAE, DAGMM, G-adVAE, E-adVAE, and adVAE), the significance level α 𝛼 \alpha was set to 0.1. Note that the best result is typeset in bold and the second best in italic typeface. The dataset names are represented as their first three letters and Avg. represents the average.

Methods	Recall						F1 Score
	Let	Car	Sat	Opt	Pen	Avg.	Let	Car	Sat	Opt	Pen	Avg.
OCSVM [31]	0.140	0.977	0.773	0.093	0.795	0.556	0.201	0.898	0.439	0.104	0.608	0.450
IForest [46]	0.160	0.830	0.827	0.407	0.987	0.642	0.215	0.816	0.539	0.401	0.700	0.534
ABOD [47]	0.750	0.983	0.893	0.940	1.000	0.913	0.739	0.647	0.657	0.709	0.674	0.685
HBOS [49]	0.200	0.625	0.853	0.587	0.827	0.618	0.256	0.703	0.490	0.507	0.620	0.515
SOD [48]	0.670	0.449	0.800	0.113	0.224	0.451	0.563	0.583	0.441	0.089	0.131	0.361
GAN [12]	0.480	0.903	0.747	0.713	0.692	0.707	0.425	0.500	0.226	0.450	0.206	0.361
MOGAAL [24]	0.180	0.455	0.493	0.367	0.276	0.354	0.277	0.578	0.627	0.455	0.319	0.451
WGAN-GP [36]	0.490	0.824	0.813	0.707	0.955	0.758	0.500	0.700	0.419	0.542	0.623	0.557
AE [13]	0.750	0.830	0.840	0.940	0.891	0.850	0.685	0.661	0.589	0.673	0.670	0.655
DAGMM [38]	0.310	0.756	0.467	0.107	0.628	0.453	0.315	0.731	0.237	0.114	0.476	0.374
VAE [13]	0.760	0.801	0.893	1.000	0.885	0.868	0.714	0.647	0.568	0.744	0.670	0.668
G-adVAE	0.690	0.960	0.787	1.000	1.000	0.887	0.717	0.824	0.549	0.718	0.721	0.706
E-adVAE	0.770	0.943	0.907	1.000	1.000	0.924	0.691	0.804	0.618	0.732	0.717	0.712
adVAE	0.780	0.938	0.853	1.000	1.000	0.914	0.729	0.805	0.631	0.718	0.704	0.717

Equations

$${\mathcal{L}}_{AE}(x,\varphi ,\theta )={\Vert x-G_{\theta }\left(E_{\varphi }(x)\right)\Vert }^2$$

$$\mathcal{L}(x)$$

$$=E_{q_{\varphi }(z|x)}\left[\log p_{\theta }(x)+\log p_{\theta }(z|x)-\log q_{\varphi }(z|x)\right]$$

$$=-KL\left(q_{\varphi }(z|x)\Vert p_{\theta }(z)\right)+E_{q_{\varphi }(z|x)}\left[\log p_{\theta }(x|z)\right]$$

$${\mathcal{L}}_{VAE}=$$

$$=$$

$${\mathcal{L}}_{MSE}(x,x_r)=$$

$${\mathcal{L}}_{KLD}(\mu ,\sigma )=$$

$$=$$

$${\mathcal{L}}_G={\mathcal{L}}_{Gz}+{\mathcal{L}}_{G{z}_T}$$

$${\mathcal{L}}_{Gz}=$$

$$=$$

$${\mathcal{L}}_{G{z}_T}=$$

$$+\gamma {\left[m_z-{\mathcal{L}}_{KLD}\left(E\left(G\left(z_T\right)\right)\right)\right]}^{+}$$

$$=$$

$$+\gamma {\left[m_z-{\mathcal{L}}_{KLD}({\mu }_{Tr},{\sigma }_{Tr})\right]}^{+}$$

$${\mathcal{L}}_T$$

$$=\int \mathcal{N}\left(z;\mu ,{\sigma }^2\right)\log \frac{\mathcal{N}\left(z;\mu ,{\sigma }^2\right)}{\mathcal{N}(z;{\mu }_T,{\sigma }_T^2)}dz$$

$$=\log \frac{{\sigma }_T}{\sigma }+\frac{{\sigma }^2+{\left(\mu -{\mu }_T\right)}^2}{2{\sigma }_T^2}-\frac{1}{2}$$

$${\mathcal{L}}_E=$$

$$+\gamma {\left[m_z-{\mathcal{L}}_{KLD}\left(E\left(G\left(z\right)\right)\right)\right]}^{+}$$

$$+\gamma {\left[m_z-{\mathcal{L}}_{KLD}\left(E\left(G\left(z_T\right)\right)\right)\right]}^{+}$$

$$=$$

$$+\gamma {\left[m_z-{\mathcal{L}}_{KLD}({\mu }_r,{\sigma }_r)\right]}^{+}$$

$$+\gamma {\left[m_z-{\mathcal{L}}_{KLD}({\mu }_{Tr},{\sigma }_{Tr})\right]}^{+}$$

$$\mathbf{S}=$$

$$=$$

$$s=\{{\mathbf{S}}_{11},{\mathbf{S}}_{22},...,{\mathbf{S}}_{nn}\}$$

$$p(s)\approx \frac{1}{mh}\sum _{i=1}^{m}K\left(\frac{s-s_i}{h}\right)$$

$$K_{RBF}(s;h)\propto \exp \left(-\frac{s^2}{2h^2}\right).$$

$$h=(\frac{m(d+2)}{4}{)}^{-\frac{1}{(d+4)}}$$

$$F(s)={\int }_{-\infty }^{s}p(s)ds$$

Full text

††journal: Knowledge-Based Systems

1 Introduction

Anomaly detection (or outlier detection) can be regarded as the task of identifying rare data items that differ from the majority of the data. Anomaly detection is applicable in a variety of domains, such as intrusion detection, fraud detection, fault detection, health monitoring, and security checking [1, 2, 3, 4, 5]. Owing to the lack of labeled anomaly samples, there is a large skew between normal and anomaly class distributions. Some attempts [6, 7, 8] use several imbalanced-learning methods to improve the performance of supervised anomaly detection models. Moreover, unsupervised models are more popular than supervised models in the anomaly detection field. Reference [9] reviewed machine-learning-based anomaly detection algorithms comprehensively.

Recently, deep generative models have become increasingly popular in anomaly detection [10]. A generative model can learn a probability distribution model by being trained on an anomaly-free dataset. Afterwards, outliers can be detected by their deviation from the probability model. The most famous deep generative models are variational autoencoders (VAEs) [11] and generative adversarial networks (GANs) [12].

VAEs have been used in many anomaly detection studies. The first work using VAEs for anomaly detection [13] declared that VAEs generalize more easily than autoencoders (AEs), because VAEs work on probabilities. [14, 15] used different types of RNN-VAE architectures to recognize the outliers of time series data. [1] and [16] implemented VAEs for intrusion detection and internet server monitoring, respectively. Furthermore, GANs and adversarial autoencoders [17] have also been introduced into image [4, 18] and video [19] anomaly detection.

However, there are two serious problems in deep-generative-model-based anomaly detection methods.

(1) Deep generative models only aim at recovering the data distribution of the training set, which has an indirect contribution to detecting anomalies. Those earlier studies paid little attention to customize their models for anomaly detection tasks. Consequently, there is an enormous problem that those models only learn from available normal samples, without attempting to discriminate the anomalous data. Owing to the lack of discrimination, it is hard for such models to learn useful deep representations for anomaly detection tasks.

(2) Plain VAEs use the regularization of the Kullback–Leibler divergence (KLD) to limit the capacity of the encoder, but there is no regularization implemented in the generator. Because neural networks are universal function approximators, generators can, in theory, cover any probability distribution, even without dependence on the latent variables [20]. However, previous attempts [21, 22] have found it hard to benefit from using an expressive and powerful generator. [20] uses Bits-Back Coding theory [23] to explain this phenomenon: if the generator can model the data distribution $p_{data}(x)$ without using information from the latent code $z$, it is more inclined not to use $z$. In this case, to reduce the KLD cost, the encoder loses much information about $x$ and maps $p_{data}(x)$ to the simple prior $p(z)$ (e.g., $\mathcal{N}(0,I)$) rather than the true posterior $p(z|x)$. Once this undesirable training phenomenon occurs, VAEs tend to overfit the distribution of existing normal data, which leads to a bad result (e.g., a high false positive rate), especially when the data are sparse. Therefore, the capacity of VAEs should be limited by an additional regularization in anomaly detection tasks.

There are only a handful of studies that attempt to solve the above two problems. MO-GAAL method [24] made the generator of a GAN stop updating before convergence in the training process and used the nonconverged generator to synthesize outliers. Hence, the discriminator can be trained to recognize outliers in a supervised manner. To try to solve the mode collapse issue of GANs, [24] expands the network structure from a single generator to multiple generators with different objectives. [25] proposed an assumption that the anomaly prior distribution is the complementary set of the normal prior distribution in latent space. However, we believe that this assumption may not hold true. If the anomalous and the normal data have complementary distributions, which means that they are separated in the latent space, then we can use a simple method (such as KNN) to detect anomalies and achieve satisfactory results, but this is not the case. Both normal data and outliers are generated by some natural pattern. Natural data ought to conform to a common data distribution, and it is hard to imagine a natural pattern that produces such a strange distribution.

To enhance deep generative models to distinguish between normal and anomalous samples and to prevent them from overfitting the given normal data, we propose a self-adversarial Variational Autoencoder (adVAE) with a Gaussian anomaly prior assumption and a self-adversarial regularization mechanism. The basic idea of this self-adversarial mechanism is adding discrimination training objectives to the encoder and the generator through adversarial training. These additional objectives will solve the above two problems at the same time; the details are as follows.

The encoder can be trained to discriminate the original sample and its reconstruction, but we do not have any anomaly latent code to train the generator. To synthesize the anomaly latent code, we propose a Gaussian anomaly hypothesis to describe the relationship between normal and anomaly latent space. Our assumption is described in Figure 1; both the anomalous and the normal prior distributions are Gaussian and have overlaps in the latent space. It is an extraordinarily weak and reasonable hypothesis, because the Gaussian distribution is the most widespread in nature. The basic structure of our self-adversarial mechanism is shown in Figure 1. The encoder is trained to discriminate the original sample $x$ and its reconstruction $x_{r}$, and the generator tries to distinguish between the normal latent variables $z$ encoded by the encoder and the anomalous ones $z_{T}$ synthesized by $T$. These new objectives we added not only give $G$ and $E$ the ability to discern, but also introduce an additional regularization to prevent model overfitting.

Our training process can be divided into two steps: (1) $T$ tries to mislead the generator $G$; meanwhile $G$ works as a discriminator. (2) $G$ generates realistic-like samples, and the encoder $E$ acts as a discriminator of $G$. To make the training phase more robust, inspired by [26], we train alternatively between the above two steps in a mini-batch iteration.

Our main contributions are summarized as follows:

We propose a novel and important concept that deep generative models should be customized to learn to discriminate outliers rather than being used in anomaly detection directly without any suitable customization. 2. 2.

We propose a novel self-adversarial mechanism, which is the prospective customization of a plain VAE, enabling both the encoder and the generator to discriminate outliers. 3. 3.

The proposed self-adversarial mechanism also provides a plain VAE with a novel regularization, which can significantly help VAEs to avoid overfitting normal data. 4. 4.

We propose a Gaussian anomaly prior knowledge assumption that describes the data distribution of anomalous latent variables. Moreover, we propose a Gaussian transformer net $T$ to integrate this prior knowledge into deep generative models. 5. 5.

The decision thresholds are automatically learned from normal data by a kernel density estimation (KDE) technique, whereas earlier anomaly detection works (e.g., [13, 16]) often omit the importance of automatic learning thresholds.

2 Preliminary

2.1 Conventional Anomaly Detection

Anomaly detection methods can be broadly categorized into probabilistic, distance-based, boundary-based and reconstruction-based.

(1) Probabilistic approach, such as GMM [27] and KDE [28], uses statistical methods to estimate the probability density function of the normal class. A data point is defined as an anomaly if it has low probability density. (2) Distance-based approach has the assumption that normal data are tightly clustered, while anomaly data occur far from their nearest neighbours. These methods depend on the well-defined similarity measure between two data points. The basic distance-based methods are LOF [29] and its modification [30]. (3) Boundary-based approach, mainly involving OCSVM [31] and SVDD [32], typically try to define a boundary around the normal class data. Whether the unknown data is an anomaly instance is determined by their location with respect to the boundary. (4) Reconstruction-based approach assumes that anomalies are incompressible and thus cannot be effectively reconstructed from low-dimensional projections. In this category, PCA [33] and its variations [34, 35] are widely used, effective techniques to detect anomalies. Besides, AE and VAE based methods also belong to this category, which will be explained detailedly in the next two subsections.

2.2 Autoencoder-based Anomaly Detection

An AE, which is composed of an encoder and a decoder, is a neural network used to learn reconstructions as close as possible to its original inputs. Given a datapoint $x\in\bm{R}^{d}$ ($d$ is the dimension of $x$), the loss function can be viewed as minimizing the reconstruction error between the training data and the outputs of the AE, and $\theta$ and $\phi$ denote the hidden parameters of the encoder $E$ and the decoder $G$:

[TABLE]

After training, the reconstruction error of each test data will be regarded as the anomaly score. The data with a high anomaly score will be defined as anomalies, because only the normal data are used to train the AE. The AE will reconstruct normal data very well, while failing to do so with anomalous data that the AE has not encountered.

2.3 VAE-based Anomaly Detection

The net architecture of VAEs is similar to that of AEs, with the difference that the encoder of VAEs forces the representation code $z$ to obey some type of prior probability distribution ${p}({z})$ (e.g., $\mathcal{N}(0,I)$). Then, the decoder generates new realistic data with code $z$ sampled from ${p}({z})$. In VAEs, both the encoder and decoder conditional distributions are denoted as $q_{\phi}({z}|{x})$ and $p_{\theta}({z}|{x})$. The data distribution $p_{\theta}({x})$ is intractable by analytic methods, and thus variational inference methods are introduced to solve the maximum likelihood $\log p_{\theta}({x})$:

[TABLE]

KLD is a similarity measure between two distributions. To estimate this maximum likelihood, VAE needs to maximize the evidence variational lower bound (ELBO) $\mathcal{L}({x})$. To optimize the KLD between $q_{\phi}(z|x)$ and $p_{\theta}(z)$, the encoder estimates the parameter vectors of the Gaussian distribution $q_{\phi}(z|x)$ : mean $\mu$ and standard deviation $\sigma$. There is an analytical expression for their KLD, because both $q_{\phi}(z|x)$ and $p_{\theta}(z)$ are Gaussian. To optimize the second term of equation (2), VAEs minimize the reconstruction errors between the inputs and the outputs. Given a datapoint $x\in\bm{R}^{d}$, the objective function can be rewritten as

[TABLE]

[TABLE]

[TABLE]

The first term $\mathcal{L}_{MSE}(x,x_{r})$ is the mean squared error (MSE) between the inputs and their reconstructions. The second term $\mathcal{L}_{KLD}(\mu,\sigma)$ regularizes the encoder by encouraging the approximate posterior $q_{\phi}(z|x)$ to match the prior $p(z)$. To hold the tradeoff between these two targets, each KLD target term is multiplied by a scaling hyperparameter $\lambda$.

AEs define the reconstruction error as the anomaly score in the test phase, whereas VAEs use the reconstruction probability [13] to detect outliers. To estimate the probabilistic anomaly score, VAEs sample $z$ according to the prior $p_{\theta}(z)$ for $L$ times and calculate the average reconstruction error as the reconstruction probability. This is why VAEs work more robustly than traditional AEs in the anomaly detection domain.

2.4 GAN-based Anomaly Detection

Since GANs [12] were first proposed in 2014, GANs have become increasingly popular and have been applied for diverse tasks. A GAN model comprises two components, which contest with each other in a cat-and-mouse game, called the generator and discriminator. The generator creates samples that resemble the real data, while the discriminator tries to recognize the fake samples from the real ones. The generator of a GAN synthesizes informative potential outliers to assist the discriminator in describing a boundary that can separate outliers from normal data effectively [24]. When a sample is input into a trained discriminator, the output of the discriminator is defined as the anomaly score. However, suffering from the mode collapsing problem, GANs usually could not learn the boundaries of normal data well, which reduces the effectiveness of GANs in anomaly detection applications. To solve this problem, [24] propose MOGAAL and suggest stopping optimizing the generator before convergence and expanding the network structure from a single generator to multiple generators with different objectives. In addition, WGAN-GP [36], one of the most advanced GAN frameworks, proposes a Wasserstein distance and gradient penalty trick to avoid mode collapsing. In our experiments, we also compared the anomaly detection performance between a plain GAN and WGAN-GP.

3 Self-adversarial Variational Autoencoder

In this section, a self-adversarial Variational Autoencoder (adVAE) for anomaly detection is proposed. To customize plain VAE to fit anomaly detection tasks, we propose the assumption of a Gaussian anomaly prior and introduce the self-adversarial mechanism into traditional VAE. The proposed method consists of three modules: an encoder net ${E}$, a generative net ${G}$, and a Gaussian transformer net ${T}$.

There are two competitive relationships in the training phase of our method: (1) To generate a potential anomalous prior distribution and enhance the generator’s ability to discriminate between normal and anomalous priors, we train the Gaussian transformer ${T}$ and the generator ${G}$ with adversarial objectives simultaneously. (2) To produce more realistic samples in a competitive manner and make the encoder learn to discern, we train the generator and the encoder analogously to the generator and discriminator in GANs.

According to equation (3), there are two components in the objective function of VAEs: $\mathcal{L}_{MSE}$ and $\mathcal{L}_{KLD}$. The cost function of adVAE is a modified combination objective of these two terms. In the following, we describe the training phase in subsections 3.1 to 3.3 and subsections 3.4 to 3 address the testing phase.

3.1 Training Step 1: Competition between T and G

The generator of plain VAE is often so powerful that it maps all the Gaussian latent code to the high-dimensional data space, even if the latent code is encoded from anomalous samples. Through the competition between ${T}$ and ${G}$, we introduce an effective regularization into the generator.

Our anomalous prior assumption suggests that it is difficult for the generator of plain VAE to distinguish the normal and the anomalous latent code, because they have overlaps in the latent space. To solve this problem, we synthesize anomalous latent variables and make the generator discriminate the anomalous from the normal latent code. As shown in Figure 2 (a), we freeze the weights of ${E}$ and update ${G}$ and ${T}$ in this training step. The Gaussian transformer ${T}$ receives the normal Gaussian latent variables $z$ encoded from the normal training samples as the inputs and transforms $z$ to the anomalous Gaussian latent variables $z_{T}$ with different mean $\mu_{T}$ and standard deviation $\sigma_{T}$. ${T}$ aims at reducing the KLD between $\{z;\mu,\sigma\}$ and $\{z_{T};\mu_{T},\sigma_{T}\}$, and ${G}$ tries to generate as different as possible samples from such two similar latent codes.

Given a datapoint $x\in\bm{R}^{d}$, the objective function in this competition process can be defined as

[TABLE]

[TABLE]

[TABLE]

[TABLE]

$[\cdot]^{+}=\max(0,\cdot)$, $m_{x}$ is a positive margin of the MSE target, and $m_{z}$ is a positive margin of the KLD target. The aim is to hold the corresponding target term below the margin value for most of the time. $\mathcal{L}_{Gz}$ is the objective for the data flow of $z$, and $\mathcal{L}_{Gz_{T}}$ is the objective for the pipeline of $z_{T}$.

$\mathcal{L}_{T}$ and $\mathcal{L}_{G}$ are two adversarial objectives, and the total objective function in this training step is the sum of the two: $\mathcal{L}_{T}+\lambda\mathcal{L}_{G}$. Objective $\mathcal{L}_{T}$ encourages $T$ to mislead $G$ by synthesizing $z_{T}$ similar to $z$, such that $G$ cannot distinguish them. Objective $\mathcal{L}_{G}$ forces the generator to distinguish between $z$ and $z_{T}$. $T$ hopes that $z_{T}$ is close to $z$, whereas $G$ hopes that $z_{T}$ is farther away from $z$. After iterative learning, $T$ and $G$ will reach a balance. $T$ will generate anomalous latent variables close to the normal latent variables, and the generator will distinguish them by different reconstruction errors. Although the anomalous latent variables synthesized by $T$ are not necessarily real, it is helpful for the models as long as they try to identify the outliers.

Because the updating of $E$ will affect the balance of $T$ and $G$, we freeze the weights of $E$ when training $T$ and $G$. If we do not do this, it will be an objective of three networks’ equilibrium, which is extremely difficult to optimize.

3.2 Training Step 2: Training E like a Discriminator

In the first training step demonstrated in the previous subsection, we freeze the weights of $E$. Instead, as shown in Figure 2 (b), we now freeze the weights of $T$ and $G$ and update the encoder $E$. The encoder not only attempts to project the data samples $x$ to Gaussian latent variables $z$ like the original VAE, but also works like a discriminator in GANs. The objective of the encoder is as follows:

[TABLE]

The first two terms of Equation 10 are the objective function of plain VAE. The encoder is trained to encode the inputs as close to the prior distribution when the inputs are from the training dataset. The last two terms are the discriminating loss we proposed. The encoder is prevented from mapping the reconstructions of training data to the latent code of the prior distribution.

The objective $\mathcal{L}_{E}$ provides the encoder with the ability to discriminate whether the input is normal because the encoder is encouraged to discover differences between the training samples (normal) and their reconstructions (anomalous). It is worth mentioning that the encoder with discriminating ability also helps the generator distinguish between the normal and the anomalous latent code.

3.3 Alternating between the Above Two Steps

As described in Algorithm 1, we train alternatively between the above two steps in a mini-batch iteration. These two steps are repeated until convergence. $detach(\cdot)$ indicates that the back propagation of the gradients is stopped at this point.

In the first training step, the Gaussian transformer converts normal latent variables into anomalous latent variables. At the same time, the generator is trained to generate realistic-like samples when the latent variables are normal and to synthesize a low-quality reconstruction when they are not normal. It offers the generator the ability to distinguish between the normal and the anomalous latent variables. In the second training step, the encoder not only maps the samples to the prior latent distribution, but also attempts to distinguish between the real data $x$ and generated samples $x_{r}$.

Importantly, we introduce the competition of $E$ and $G$ into our adVAE model by training alternatively between these two steps. Analogously to GANs, the generator is trained to fool the encoder in training step 1, and the encoder is encouraged to discriminate the samples generated by the generator in step 2. In addition to benefitting from adversarial alternative learning as in GANs, the encoder and generator models will also learn jointly for the given training data to maintain the advantages of VAEs.

3.4 Anomaly Score

As demonstrated in Figure 2 (c), only the generator and the encoder are used in the testing phase, as in a traditional VAE. Given a test data point $x\in\bm{R}^{d}$ as the input, the encoder estimates the parameters of the latent Gaussian variables $\mu$ and $\sigma$ as the output. Then, the reparameterization trick is used to sample $\bm{z}=\{z^{(1)},z^{(2)},...,z^{(L)}\}$ according to the latent distribution $\mathcal{N}\left(\mu,\sigma^{2}\right)$ , i.e., $z^{(l)}=\mu+\sigma\odot\varepsilon^{(l)}$, where $\varepsilon\sim\mathcal{N}(0,I)$ and $l=1,2,...L$. $L$ is set to 1000 in this work and used to improve the robustness of adVAE’s performance. The generator receives $z^{(l)}$ as the input and outputs the reconstruction $x_{r}^{(l)}\in\bm{R}^{d}$.

The error between the inputs $x$ and their average reconstruction $\sum^{L}_{l=1}x_{r}^{(l)}$ reflects the deviation between the testing data and the normal data distribution learned by adVAE, such that the anomaly score of a mini-batch data $\bm{x}\in\bm{R}^{n\times d}$ ($n$ is the batch size) is defined as follows:

[TABLE]

[TABLE]

with error matrix $\bm{e}\in\bm{R}^{n\times d}$ and squared error matrix $\bm{S}\in\bm{R}^{n\times n}$. $s\in\bm{R}^{n}$ is the anomaly scores vector of a mini-batch dataset $\bm{x}$. The dimension of $s$ is always equal to the batch size of $\bm{x}$. Data points with a high anomaly score are classified as anomalies. To determine whether an anomaly score is high enough, we need a decision threshold. In the next subsection, we will illustrate how to decide the threshold automatically by KDE [37].

3.5 Decision Threshold

Earlier VAE-based anomaly detection studies [13, 16] often overlook the importance of threshold selection. Because we have no idea about what the values of reconstruction error actually represent, determining the threshold of reconstruction error is cumbersome. Some studies [13, 14, 15] adjust the threshold by cross-validation. However, building a big enough validation set is a luxury in some cases, as anomalous samples are challenging to collect. Other attempts [16, 38] simply report the best performance in the test dataset to evaluate models, which makes it difficult to reproduce the results in practical applications. Thus, it is exceedingly important to let the anomaly detection model automatically determine the threshold.

The KDE technique [37] is used to determine the decision threshold in the case where only normal samples are provided. Given the anomaly scores vector $s$ of the training dataset, KDE estimates the probability density function (PDF) $p(s)$ in a nonparametric way:

[TABLE]

where $m$ is the size of the training dataset, $\{s_{i}\}$, $i=1,2...,m$, is the training dataset’s anomaly scores vector, $K(\cdot)$ is the kernel function, and $h$ is the bandwidth.

Among all kinds of kernel functions, radial basis functions (RBFs) are the most commonly used in density estimation. Therefore, a RBF kernel is used to estimate the PDF of the normal training dataset:

[TABLE]

In practice, the choice of the bandwidth parameter $h$ has a significant influence on the effect of the KDE. To make a good choice of bandwidth parameters, Silverman [39] derived a selection criterion:

[TABLE]

where $m$ is the number of training data points and $d$ the number of dimensions. After obtaining the PDF $p(s)$ of the training dataset by KDE, the cumulative distribution function (CDF) $F(s)$ can be obtained by equation (16):

[TABLE]

Given a significance level $\alpha\in[0,1]$ and a CDF, we can find a decision threshold $s_{\alpha}$ that satisfies $F(s_{\alpha})=1-\alpha$. In this case, there is at least $(1-\alpha)100\%$ probability that a sample with the anomaly score $s\geq s_{\alpha}$ is an outlier. Because KDE decides the threshold by estimating the interval of normal data’s anomaly scores, it is clear that using KDE to decide the threshold is more objective and reasonable than simply relying on human experience. A higher significance level $\alpha$ leads to a lower missing alarm rate, which means that models have fewer chances to mislabel outliers as normal data. On the contrary, a lower $\alpha$ means a lower false alarm rate. Therefore, the choice of the significance level $\alpha$ is a tradeoff. The significance level $\alpha$ is recommended to be set to 0.1 for anomaly detection tasks.

3.6 Detecting Outliers

In this subsection, we summarize how to use a trained adVAE model and the KDE technique to learn the threshold from a training dataset and detect outliers in a testing dataset. As illustrated in Figure 3, this process is divided into two parts.

Part I focuses on the training process. Given a training dataset matrix $X\in\bm{R}^{m\times d}$ consisting of normal samples, we can calculate their anomaly scores vector $s\in\bm{R}^{m}$ (described in subsection 3.4), where $m$ is the size of training dataset and $d$ is the dimension of the data. As described in subsection 3.5, the PDF $p(s)$ of the anomaly scores vector is obtained by KDE and the CDF $F(s)$ is obtained by $\int^{s}_{-\infty}p(s)ds$. Then, the decision threshold $s_{\alpha}$ can be determined from a given significance level $\alpha$ and CDF $F(s)$.

Part II is simple and easy to understand. The anomaly score $s_{new}$ of a new sample $x_{new}$ is calculated by adVAE. If $s_{new}\geq s_{\alpha}$, then $x_{new}$ is defined as an outlier. If not, then $x_{new}$ is regarded as a normal sample.

4 Experiments

4.1 Datasets

Most previous works used image datasets to test their anomaly detection models. To eliminate the impact of different convolutional structures and other image tricks on the test performance, we chose five publicly available and broadly used tabular anomaly detection datasets to evaluate our adVAE model. All the dataset characteristics are summarized in Table 1. For each dataset, 80% of the normal data were used for the training phase, and then the remaining 20% and all the outliers were used for testing. More details about the datasets can be found in their references or our Github repository222https://github.com/WangXuhongCN/adVAE.

4.2 Evaluation Metric

The anomaly detection community defines anomalous samples as positive and defines normal samples as negative, hence the anomaly detection tasks can also be regarded as a two-class classification problem with a large skew in class distribution. For the evaluation of a two-class classifier, the metrics are divided into two categories; one category is defined at a single threshold, and the other category is defined at all possible thresholds.

Metrics at a single threshold. Accuracy, precision, recall, and the F1 score are the common metrics to evaluate models performance at a single threshold. Because the class distribution is skew, accuracy is not a suitable metric for anomaly detection model evaluation.

High precision means the fewer chances of misjudging normal data, and a high recall means the fewer chances of models missing alarming outliers. Even if models predict a normal sample as an outlier, people can still correct the judgment of the model through expert knowledge, because the anomalous samples are of small quantity. However, if models miss alarming outliers, we cannot find anomalous data in such a huge dataset. Thus, precision is not as crucial as recall. The F1 score is the harmonic average of precision and recall. Therefore, we adopt recall and the F1 score as the metrics for comparing at a single threshold.

Metrics at all possible thresholds. The anomaly detection community often uses receiver operator characteristic (ROC) and precision–recall (PR) curves, which aggregate over all possible decision thresholds, to evaluate the predictive performance of each method. When the class distribution is close to being uniform, ROC curves have many desirable properties. However, because anomaly detection tasks always have a large skew in the class distribution, PR curves give a more accurate picture of an algorithm’s performance [45].

Rather than comparing curves, it is useful and clear to analyze the model performance quantitatively using a single number. Average precision (AP) and area under the ROC curve (AUC) are the common metrics to measure performance, with the former being preferred under class imbalance.

AP summarizes a PR curve by a sum of precisions at each threshold, multiplied by the increase in recall, which is a close approximation of the area under the PR curve: $AP=\sum_{n}P_{n}\Delta R_{n}$, where $P_{n}$ is the precision at the $n^{th}$ threshold and $\Delta R_{n}$ is the increase in recall from the $n-1^{th}$ to the $n^{th}$ threshold. Because the PR curve is more useful than the ROC curve in anomaly detection, we recommend using AP as an evaluation metric for anomaly detection models rather than AUC. In our experiments, recall, F1 score, AUC, and AP were used to evaluate the models’ performance.

4.3 Model Configurations

The network structures of adVAE used are summarized in Figure 4, where the size of each layer was proportionate to the input dimension $dim(x)$. The Adam optimizer was used in all datasets with a learning rate of 0.001. We trained each model for a maximum 20000 mini-batch iterations with a batch size of 128. Kaiming uniform weight initialization was used for all three subnetworks.

As shown in Table 2, there are four hyperparameters in adVAE model. $\lambda$ is derived from plain VAE, and the other three new parameters are used to maintain the balance between the original training objective and the additional discrimination objective. The larger the $\gamma$ or $m_{z}$, the larger the proportion of the encoder discrimination objective in the total loss. The larger the $m_{x}$, the larger the proportion of the generator discrimination objective.

The KLD margin $m_{z}$ can be selected to be slightly larger than the training KLD value of plain VAEs, and the MSE margin $m_{x}$ was set equal to 2 for each dataset. Because $\gamma$ and $m_{z}$ have similar effects, $\gamma$ is always suggested to be fixed as 0.001. Adjusting the parameters becomes easier, because we can simply consider tuning $m_{x}$ and $m_{z}$. Note that the structure and hyperparameters used in this study proved to be sufficient for our applications, although they can still be improved.

4.4 Compared Methods

We compare adVAE with the eleven most advanced outlier detection algorithms, which include both traditional and deep-learning-based methods. To obtain a convincing conclusion, the hyperparameters of competing methods are searched in a range of optional values. They can be divided into seven categories: (i) A boundary-based approach, OCSVM [31]. It is a widely used semisupervised outlier detection algorithm, and we use a RBF kernel in all tasks. Because OCSVM needs an important parameter $\nu$, its value will be searched in the range {0.01, 0.05, 0.1, 0.15, 0.2}. (ii) An ensemble-learning-based approach, IForest [46]. The hyperparameter to be tuned is the number of decision trees $n_{t}$, which is chosen from the set {50, 100, 200, 300}. (iii) A probabilistic approach, ABOD [47]. (iv) Distance-based approaches, SOD [48] and HBOS [49]. Because the performance of ABOD and SOD will be dramatically affected by the size of the neighborhood, we tune it in the range of {5, 8, 16, 24, 40}. For HBOS, the number of bins is chosen from {5, 10, 20, 30, 50}. (v) Three GAN-based models, GAN [12], MOGAAL [24], and WGAN-GP [36]. GAN shares the same network structure as adVAE, except that the output layer of the discriminator is one-dimensional, and the configurations of MOGAAL333https://github.com/leibinghe/GAAL-based-outlier-detection refer to its official open source code. Based on the network structure of GAN, WGAN-GP removes the output activation function of the generator. (vi) Three reconstruction-based methods, AE [13], DAGMM [38], and VAE [13]. The AE, VAE, and DAGMM share the same network structure as adVAE, except that the latent dimension of DAGMM444https://github.com/danieltan07/dagmm is required to be 1. VAE shares the same hyperparameter $\lambda$ as adVAE. (vii) Two ablation models of the proposed adVAE, named E-adVAE and G-adVAE, are also compared to demonstrate how the two discrimination objectives affect the performance. Figure 5 and 6 show the architecture of our two ablation models, respectively. E-adVAE, G-adVAE, and adVAE share the same parameters and network structure.

All deep-learning methods are implemented in pytorch, and share the same optimizer, learning rate, batch size, and iteration times as adVAE, except that the parameters of DAGMM and MOGAAL refer to their author’s recommendation. All traditional methods are implemented on a common outlier detection framework PyOD [50].

4.5 Results and Discussion

Table 3 shows the experimental results for the AP and AUC metrics, and Table 4 indicates the results for the recall and F1 score. According to the experimental results, the adVAE model achieved 11 best and 5 second-best results from 24 comparisons. Therefore, adVAE is significantly better than other compared methods.

Among all the experimental results, the performance of VAE is generally slightly better than AE, because VAE has a KLD regularization of the encoder, which proves the importance of regularization in AE-based anomaly detection algorithms.

We customize plain VAE by the proposed self-adversarial mechanism, reaching the highest AP improvement of 0.255 (40.8%). Compared to its base technique VAE, on average, adVAE improved by 0.113 (14.9%) in AP, 0.036 (3.9%) in AUC, 0.046 (5.3%) in recall, and 0.049 (7.3%) in F1 score. The outstanding results of adVAE prove the superiority of the proposed self-adversarial mechanism, which helps deep generative models to play a greater role in anomaly detection research and applications.

OCSVM performed well in the cardio, satellite, and pen datasets, but achieved horrible results in the letter and optical datasets. The results indicate that the conventional method OCSVM is not stable enough, leading the AP gap from adVAE to reach an astonishing 0.8 in the optical dataset. Actually, all traditional anomaly detection methods (e.g., IForest, HBOS, and SOD) have the same problem as OCSVM: They perform well on some datasets, but they perform extremely poorly on other datasets. ABOD’s performance is more stable than others, but it still suffers from the above problem.

The reason for this is that conventional methods often have a strict data assumption. If training data are high-dimensional or not consistent with the data assumption, their performance may be significantly impaired. Because neural-network-based methods do not need a strict data assumption and are suitable for big and high-dimensional data, they have more stable performance on diverse datasets with different data distributions.

In our experiments, both VAE and AE outperformed the GAN-based methods in most datasets. The results are analyzed as follows: Because the mode collapse issue of GANs cannot be characterized in a training loss curve, people usually monitor the generated images during the training phase. However, for tabular datasets, it is difficult for us to ensure that the generated samples are diverse enough. A GAN with mode collapse would mislabel normal data as outliers and cause a high false positive rate, because GAN learns an incomplete normal data distribution in this case. To better recover the distribution of normal data, MOGAAL suggests stopping optimizing the generator before convergence and expanding the network structure from a single generator to multiple generators with different objectives. However, a nonconverged model brings more uncertainty to experimental results, because it is impossible to know what the model has learned precisely. According to the results, it does have a performance improvement over plain GAN, but still has a performance gap with AE-based methods. WGAN-GP tries to avoid mode collapsing, which leads to better results of all four metrics than plain GAN. However, it still results in worse performance than encoder-decoder-based methods. GAN is better at generating high-quality samples rather than learning the complete probability distribution of the training dataset. Thus GAN-based methods do not perform well for outlier detection.

For three reconstruction-based neural network models, there is no significant performance gap between AE and VAE, whereas DAGMM is worse than them in our tests. This is explained by the fact that the latent dimension of DAGMM is required to be set as 1, which severely limits the ability of the model to summarize datasets. The results also indicate this explanation: DAGMM only achieves good results in the cardio dataset, which has the lowest data size and dimensions. The larger the size and dimensions of the dataset, the larger performance gap between DAGMM and AE.

In conclusion, the AE and VAE methods are better than most of the other anomaly detection methods according to our tests. Moreover, adVAE learns to detect outliers through the self-adversarial mechanism, which further enhances the anomaly detection capability and achieves state-of-the-art performance. We believe that the proposed self-adversarial mechanism has immense potentiality, and it deserves to be extended to other pattern recognition tasks.

4.6 Ablation Study

To illustrate the impact of different designs on the model results, we design three different types of ablation experiments:

(i) Because adVAE is formed by adding two different training objectives to plain VAE, it is important to demonstrate how the two discrimination objectives affect the performance. Hence we propose E-adVAE and G-adVAE models (Figure 5 and 6) to demonstrate the effect of removing one of the discrimination objectives from adVAE. According to Table 3 and 4, the results show that both G-adVAE and E-adVAE achieve performance improvements compared to plain VAE, which indicates the validity of the two discriminative objectives of adVAE. In addition, E-adVAE is more competitive than G-adVAE in most datasets. This result is analyzed as follows. Because the latent variables $z$ have a lower dimension than the original data $x$, $x$ has more information than $z$. Therefore, the benefit of adding the discriminating target to the encoder $E$ is more significant than adding it to the generator $G$.

(ii) Because the encoder and generator work together to detect outliers, we performed a latent space visualization to prove the improvement in the encoder’s anomaly detection capability. Figure 8 illustrates that the encoder of adVAE has a better discrimination ability, because the latent variables are separated in the latent space and more easily distinguished by the later generator.

(iii) To independently verify the anomaly detection capabilities of the generator, we synthesize latent outlier variables $z_{o}$ by adding five types of noise to the normal latent variables $z_{n}$. These latent variables are the outputs of the five corresponding encoders and the input data of the encoders are 200 normal samples $x_{n}$ from the pen [44] dataset. The normal scores vector $s_{n}\in\bm{R}^{200}$ can be calculated from $x_{n}$ and $G(z_{n})$, and the anomalous scores vector $s_{o}\in\bm{R}^{200}$ can be calculated from $x_{n}$ and $G(z_{o})$ (as in section 3.4). Afterwards, the Wasserstein distance between $s_{n}$ and $s_{o}$ is used to measure the detection capabilities of each generator. Figure 7 indicates that the generator $G$ of adVAE also has a better ability to discriminate latent variables.

In conclusion, benefiting from better discrimination abilities, adVAE has better anomaly detection performance. This proves that the proposed self-adversarial mechanism is a prospective way of customizing generative models to fit outlier detection tasks.

4.7 Robustness Experiments

Because adVAE involves four primary hyperparameters, it is essential to implement in-depth hyperparameter experiments comprehensively. As shown in Figure 9(a)–9(d), we tested the results for four hyperparameters. Actually, only $m_{z}$ and $m_{x}$ need to be tuned, because $\lambda$ is derived from VAE and $\gamma$ is fixed as 0.01. Based on that, finding good hyperparameters of adVAE is not more complicated than plain VAE, because it is not sensitive to $m_{z}$ and $m_{x}$ according to Figure 9(c) and 9(d).

As for the learning ability of neural networks, we tune the number of layers from 2 to 6 and adjust the number of hidden neurons from $\frac{1}{3}dim(x)$ to $dim(x)$. The results are shown in Figure 9(e), where V1–V5 represent the parameter variation. It can be seen that V1 achieved a slightly worse detection capability, because the insufficient networks cannot learn enough information of the normal training data. As long as the scale of neural network is within a reasonable range, adVAE is robust to network structure.

Because adVAE needs an anomaly-free dataset, it is important to investigate how adVAE responds to contaminated training data. Meanwhile, we also choose three semisupervised anomaly detection methods for comparison. ADASYN [51] is used to synthesize new anomalous samples. The results of Figure 9(f) show that OCSVM is more robust with a contaminated dataset than the other three AE-based methods. This is because OCSVM can ignore certain noises when learning the boundary of the training data, whereas AE-based models always try to reduce the reconstruction error for all training data. Moreover, a high contamination ratio will more easily disrupt the proposed discrimination loss, which suggests training an AE-based anomaly detection model with high-quality data (i.e., a clean or low-contamination-ratio dataset). In practice, normal data is easy to collect, and thus the contamination ratio usually remains at a low level.

5 Conclusion

In this paper, we have proposed a self-adversarial Variational Auto-encoder (adVAE) with a Gaussian anomaly prior assumption and a self-adversarial mechanism. The proposed adVAE is encouraged to distinguish the normal latent code and the anomalous latent variables generated by the Gaussian transformer $T$, which can also be regarded as an outstanding regularization introduced into VAE-based outlier detection method. The results demonstrate that the proposed adVAE outperforms than other state-of-the-art anomaly detection methods. In the future, we will try to redesign the discrimination objective of the generator to further enhance the generator’s ability to recognize anomalies.

Acknowledgement

This research is in part supported by National Nature Science Foundation of China (No. 51777122).