Weighting vectors for machine learning: numerical harmonic analysis applied to boundary detection

TL;DR

This paper explores the use of weighting vectors derived from metric space magnitude for boundary detection and outlier detection, linking them to kernelized SVMs and demonstrating competitive performance with efficient approximations.

Contribution

It introduces a novel interpretation of weighting vectors as solutions to kernelized SVMs and proposes efficient linear-time approximations for boundary and outlier detection.

Findings

Weighting vectors effectively detect boundaries in Euclidean spaces.

The approach outperforms or matches state-of-the-art outlier detection methods.

Linear-time approximation methods are feasible for large datasets.

Abstract

Metric space magnitude, an active field of research in algebraic topology, is a scalar quantity that summarizes the effective number of distinct points that live in a general metric space. The {\em weighting vector} is a closely-related concept that captures, in a nontrivial way, much of the underlying geometry of the original metric space. Recent work has demonstrated that when the metric space is Euclidean, the weighting vector serves as an effective tool for boundary detection. We recast this result and show the weighting vector may be viewed as a solution to a kernelized SVM. As one consequence, we apply this new insight to the task of outlier detection, and we demonstrate performance that is competitive or exceeds performance of state-of-the-art techniques on benchmark data sets. Under mild assumptions, we show the weighting vector, which has computational cost of matrix inversion, can be efficiently approximated in linear time. We show how nearest neighbor methods can approximate solutions to the minimization problems defined by SVMs.