Constant Time EXPected Similarity Estimation using Stochastic Optimization

TL;DR

This paper introduces an improved version of the EXPected Similarity Estimation (EXPoSE) algorithm that achieves constant-time prediction accuracy for large-scale anomaly detection by using stochastic optimization.

Contribution

It reformulates EXPoSE as a stochastic optimization problem, enabling epsilon-accurate models to be estimated in constant time regardless of dataset size.

Findings

Achieves epsilon-accurate models in constant time

Applicable to infinite-dimensional Hilbert spaces

No additional step-size parameters needed

Abstract

A new algorithm named EXPected Similarity Estimation (EXPoSE) was recently proposed to solve the problem of large-scale anomaly detection. It is a non-parametric and distribution free kernel method based on the Hilbert space embedding of probability measures. Given a dataset of $n$ samples, EXPoSE needs only $\mathcal{O}(n)$ (linear time) to build a model and $\mathcal{O}(1)$ (constant time) to make a prediction. In this work we improve the linear computational complexity and show that an $\epsilon$-accurate model can be estimated in constant time, which has significant implications for large-scale learning problems. To achieve this goal, we cast the original EXPoSE formulation into a stochastic optimization problem. It is crucial that this approach allows us to determine the number of iteration based on a desired accuracy $\epsilon$, independent of the dataset size $n$. We will show that the proposed stochastic gradient descent algorithm works in general (possible infinite-dimensional) Hilbert spaces, is easy to implement and requires no additional step-size parameters.