Active Learning of SVDD Hyperparameter Values

TL;DR

This paper introduces LAMA, a novel active learning method for accurately estimating SVDD hyperparameters, reducing manual validation and outperforming existing heuristics in outlier detection tasks.

Contribution

LAMA is the first principled, evidence-based approach to estimate both SVDD hyperparameters using active learning and kernel alignment, with a quality score included.

Findings

LAMA outperforms state-of-the-art methods in real-world experiments.

LAMA provides hyperparameter estimates close to empirical upper bounds.

Estimates are evidence-based and eliminate manual validation.

Abstract

Support Vector Data Description is a popular method for outlier detection. However, its usefulness largely depends on selecting good hyperparameter values -- a difficult problem that has received significant attention in literature. Existing methods to estimate hyperparameter values are purely heuristic, and the conditions under which they work well are unclear. In this article, we propose LAMA (Local Active Min-Max Alignment), the first principled approach to estimate SVDD hyperparameter values by active learning. The core idea bases on kernel alignment, which we adapt to active learning with small sample sizes. In contrast to many existing approaches, LAMA provides estimates for both SVDD hyperparameters. These estimates are evidence-based, i.e., rely on actual class labels, and come with a quality score. This eliminates the need for manual validation, an issue with current heuristics. LAMA outperforms state-of-the-art competitors in extensive experiments on real-world data. In several cases, LAMA even yields results close to the empirical upper bound.