A simple efficient density estimator that enables fast systematic search

TL;DR

This paper presents a new simple and efficient density estimator that significantly speeds up systematic subspace search in outlying aspects mining, enabling analysis of large, high-dimensional datasets.

Contribution

The paper introduces a novel density estimator that improves computational efficiency, allowing existing outlying aspects mining methods to handle larger and higher-dimensional datasets.

Findings

The new estimator is faster than kernel density estimators.

Replacing the density estimator in existing methods greatly increases speed.

The approach enables analysis of datasets with thousands of dimensions.

Abstract

This paper introduces a simple and efficient density estimator that enables fast systematic search. To show its advantage over commonly used kernel density estimator, we apply it to outlying aspects mining. Outlying aspects mining discovers feature subsets (or subspaces) that describe how a query stand out from a given dataset. The task demands a systematic search of subspaces. We identify that existing outlying aspects miners are restricted to datasets with small data size and dimensions because they employ kernel density estimator, which is computationally expensive, for subspace assessments. We show that a recent outlying aspects miner can run orders of magnitude faster by simply replacing its density estimator with the proposed density estimator, enabling it to deal with large datasets with thousands of dimensions that would otherwise be impossible.