Universal Outlying sequence detection For Continuous Observations

TL;DR

This paper develops a universal KL divergence-based test for detecting an outlier sequence among multiple continuous data sequences, even when the outlier distribution is unknown, and compares its performance with an MMD-based test.

Contribution

It introduces a new universal test using KL divergence with exponential consistency guarantees for outlier detection in continuous data sequences.

Findings

The KL divergence estimator converges exponentially fast under certain density ratio conditions.

The proposed test outperforms MMD-based methods in specific regimes.

The test is applicable when the outlier distribution is unknown but bounded relative to the typical distribution.

Abstract

The following detection problem is studied, in which there are $M$ sequences of samples out of which one outlier sequence needs to be detected. Each typical sequence contains $n$ independent and identically distributed (i.i.d.) continuous observations from a known distribution $\pi$, and the outlier sequence contains $n$ i.i.d. observations from an outlier distribution $\mu$, which is distinct from $\pi$, but otherwise unknown. A universal test based on KL divergence is built to approximate the maximum likelihood test, with known $\pi$ and unknown $\mu$. A data-dependent partitions based KL divergence estimator is employed. Such a KL divergence estimator is further shown to converge to its true value exponentially fast when the density ratio satisfies $0<K_1\leq \frac{d\mu}{d\pi}\leq K_2$, where $K_1$ and $K_2$ are positive constants, and this further implies that the test is exponentially consistent. The performance of the test is compared with that of a recently introduced test for this problem based on the machine learning approach of maximum mean discrepancy (MMD). We identify regimes in which the KL divergence based test is better than the MMD based test.