Neural Network Training Strategy to Enhance Anomaly Detection Performance: A Perspective on Reconstruction Loss Amplification

TL;DR

This paper introduces LAMP, a simple yet effective training strategy that amplifies the reconstruction loss to sharpen the loss landscape, thereby improving anomaly detection performance without altering neural network architecture.

Contribution

The paper proposes LAMP, a novel loss amplification method that enhances anomaly detection by shaping the loss landscape, independent of network design changes.

Findings

LAMP increases the steepness of the loss landscape.

LAMP improves anomaly detection accuracy.

LAMP is applicable with various reconstruction error metrics.

Abstract

Unsupervised anomaly detection (UAD) is a widely adopted approach in industry due to rare anomaly occurrences and data imbalance. A desirable characteristic of an UAD model is contained generalization ability which excels in the reconstruction of seen normal patterns but struggles with unseen anomalies. Recent studies have pursued to contain the generalization capability of their UAD models in reconstruction from different perspectives, such as design of neural network (NN) structure and training strategy. In contrast, we note that containing of generalization ability in reconstruction can also be obtained simply from steep-shaped loss landscape. Motivated by this, we propose a loss landscape sharpening method by amplifying the reconstruction loss, dubbed Loss AMPlification (LAMP). LAMP deforms the loss landscape into a steep shape so the reconstruction error on unseen anomalies becomes greater. Accordingly, the anomaly detection performance is improved without any change of the NN architecture. Our findings suggest that LAMP can be easily applied to any reconstruction error metrics in UAD settings where the reconstruction model is trained with anomaly-free samples only.