Infinite-dimensional Mahalanobis Distance with Applications to Kernelized Novelty Detection

TL;DR

This paper extends Mahalanobis distance to infinite-dimensional Banach spaces, providing a basis-free, data-driven anomaly measure applicable to kernelized settings and demonstrating improved novelty detection in time series data.

Contribution

It introduces a basis-free, covariance-invariant Mahalanobis distance in Banach spaces, connecting it to RKHS and developing a kernelized nearest-neighbour variant for novelty detection.

Findings

Variance norm is invariant under invertible transformations.

The proposed method outperforms traditional Mahalanobis distance in time series novelty detection.

The framework generalizes classical and kernelized Mahalanobis distances to infinite-dimensional spaces.

Abstract

The Mahalanobis distance is a classical tool used to measure the covariance-adjusted distance between points in $\bbR^d$. In this work, we extend the concept of Mahalanobis distance to separable Banach spaces by reinterpreting it as a Cameron-Martin norm associated with a probability measure. This approach leads to a basis-free, data-driven notion of anomaly distance through the so-called variance norm, which can naturally be estimated using empirical measures of a sample. Our framework generalizes the classical $\bbR^d$, functional $(L^2[0,1])^d$, and kernelized settings; importantly, it incorporates non-injective covariance operators. We prove that the variance norm is invariant under invertible bounded linear transformations of the data, extending previous results which are limited to unitary operators. In the Hilbert space setting, we connect the variance norm to the RKHS of the covariance operator, and establish consistency and convergence results for estimation using empirical measures with Tikhonov regularization. Using the variance norm, we introduce the notion of a kernelized nearest-neighbour Mahalanobis distance, and study some of its finite-sample concentration properties. In an empirical study on 12 real-world data sets, we demonstrate that the kernelized nearest-neighbour Mahalanobis distance outperforms the traditional kernelized Mahalanobis distance for multivariate time series novelty detection, using state-of-the-art time series kernels such as the signature, global alignment, and Volterra reservoir kernels.