An Efficient Variant of One-Class SVM with Lifelong Online Learning Guarantees

TL;DR

This paper introduces SONAR, an efficient SGD-based one-class SVM method with theoretical guarantees for outlier detection in non-stationary streaming data, improving accuracy and adaptability over traditional approaches.

Contribution

The paper proposes SONAR, a novel SGD-based one-class SVM with strong theoretical guarantees for non-stationary data, including lifelong learning and adversarial robustness.

Findings

SONAR achieves lower false-negative rates than traditional OCSVM.

Theoretical guarantees improve error bounds under non-stationarity.

Ensemble and changepoint detection enhance robustness in adversarial settings.

Abstract

We study outlier (a.k.a., anomaly) detection for single-pass non-stationary streaming data. In the well-studied offline or batch outlier detection problem, traditional methods such as kernel One-Class SVM (OCSVM) are both computationally heavy and prone to large false-negative (Type II) errors under non-stationarity. To remedy this, we introduce SONAR, an efficient SGD-based OCSVM solver with strongly convex regularization. We show novel theoretical guarantees on the Type I/II errors of SONAR, superior to those known for OCSVM, and further prove that SONAR ensures favorable lifelong learning guarantees under benign distribution shifts. In the more challenging problem of adversarial non-stationary data, we show that SONAR can be used within an ensemble method and equipped with changepoint detection to achieve adaptive guarantees, ensuring small Type I/II errors on each phase of data. We validate our theoretical findings on synthetic and real-world datasets.