LOSDD: Leave-Out Support Vector Data Description for Outlier Detection

TL;DR

LOSDD introduces a leave-out strategy for SVM-based outlier detection that improves accuracy on dirty data by iteratively identifying and removing outliers, reducing masking effects, and optimizing training efficiency.

Contribution

The paper proposes a novel leave-out approach for SVDD that enhances outlier detection in contaminated data and offers an efficient incremental training method.

Findings

Effective outlier detection in dirty data

Reduction of masking effects in outlier identification

Incremental training accelerates the leave-out SVM process

Abstract

Support Vector Machines have been successfully used for one-class classification (OCSVM, SVDD) when trained on clean data, but they work much worse on dirty data: outliers present in the training data tend to become support vectors, and are hence considered "normal". In this article, we improve the effectiveness to detect outliers in dirty training data with a leave-out strategy: by temporarily omitting one candidate at a time, this point can be judged using the remaining data only. We show that this is more effective at scoring the outlierness of points than using the slack term of existing SVM-based approaches. Identified outliers can then be removed from the data, such that outliers hidden by other outliers can be identified, to reduce the problem of masking. Naively, this approach would require training N individual SVMs (and training $O(N^2)$ SVMs when iteratively removing the worst outliers one at a time), which is prohibitively expensive. We will discuss that only support vectors need to be considered in each step and that by reusing SVM parameters and weights, this incremental retraining can be accelerated substantially. By removing candidates in batches, we can further improve the processing time, although it obviously remains more costly than training a single SVM.