A Radon-Nikod\'ym Perspective on Anomaly Detection: Theory and Implications

TL;DR

This paper introduces RN-Loss, a novel anomaly detection loss function derived from Radon-Nikodým theorem, which improves performance by weighting the loss with the Radon-Nikodým derivative, validated across diverse datasets.

Contribution

It applies Radon-Nikodým derivatives to design a new loss function for anomaly detection, with theoretical proof and extensive empirical validation.

Findings

Outperforms state-of-the-art on 68% of multivariate datasets

Achieves peak F1-scores on 72% of univariate time series datasets

Demonstrates broad applicability across healthcare, cybersecurity, and finance

Abstract

Which principle underpins the design of an effective anomaly detection loss function? The answer lies in the concept of Radon-Nikod\'ym theorem, a fundamental concept in measure theory. The key insight from this article is: Multiplying the vanilla loss function with the Radon-Nikod\'ym derivative improves the performance across the board. We refer to this as RN-Loss. We prove this using the setting of PAC (Probably Approximately Correct) learnability. Depending on the context a Radon-Nikod\'ym derivative takes different forms. In the simplest case of supervised anomaly detection, Radon-Nikod\'ym derivative takes the form of a simple weighted loss. In the case of unsupervised anomaly detection (with distributional assumptions), Radon-Nikod\'ym derivative takes the form of the popular cluster based local outlier factor. We evaluate our algorithm on 96 datasets, including univariate and multivariate data from diverse domains, including healthcare, cybersecurity, and finance. We show that RN-Derivative algorithms outperform state-of-the-art methods on 68% of Multivariate datasets (based on F1 scores) and also achieves peak F1-scores on 72% of time series (Univariate) datasets.