Computationally Efficient Estimators for Dimension Reductions Using Stable Random Projections

TL;DR

This paper introduces an optimal quantile estimator for stable random projections that significantly reduces computational costs while maintaining or improving accuracy in estimating $l_eta$ distances, especially for large-scale data.

Contribution

It proposes a new, computationally efficient quantile-based estimator with proven theoretical accuracy and explicit error bounds, outperforming previous estimators in large-scale applications.

Findings

Nearly one order of magnitude faster than previous estimators.

More accurate than previous estimators when $eta > 1$.

Provides explicit error bounds and sample complexity analysis.

Abstract

The method of stable random projections is a tool for efficiently computing the $l_\alpha$ distances using low memory, where $0<\alpha \leq 2$ is a tuning parameter. The method boils down to a statistical estimation task and various estimators have been proposed, based on the geometric mean, the harmonic mean, and the fractional power etc. This study proposes the optimal quantile estimator, whose main operation is selecting, which is considerably less expensive than taking fractional power, the main operation in previous estimators. Our experiments report that the optimal quantile estimator is nearly one order of magnitude more computationally efficient than previous estimators. For large-scale learning tasks in which storing and computing pairwise distances is a serious bottleneck, this estimator should be desirable. In addition to its computational advantages, the optimal quantile estimator exhibits nice theoretical properties. It is more accurate than previous estimators when $\alpha>1$. We derive its theoretical error bounds and establish the explicit (i.e., no hidden constants) sample complexity bound.