Asymmetric Random Projections

TL;DR

This paper introduces a data-dependent random projection method that leverages data statistics to improve the accuracy of dimensionality reduction in tasks like matrix multiplication and regression, outperforming traditional oblivious methods.

Contribution

It proposes a novel, computationally efficient data-dependent random projection technique that enhances performance in various linear algebra and machine learning tasks.

Findings

Significant improvements in matrix multiplication speed.

Enhanced accuracy in linear regression and classification.

Outperforms traditional oblivious random projections.

Abstract

Random projections (RP) are a popular tool for reducing dimensionality while preserving local geometry. In many applications the data set to be projected is given to us in advance, yet the current RP techniques do not make use of information about the data. In this paper, we provide a computationally light way to extract statistics from the data that allows designing a data dependent RP with superior performance compared to data-oblivious RP. We tackle scenarios such as matrix multiplication and linear regression/classification in which we wish to estimate inner products between pairs of vectors from two possibly different sources. Our technique takes advantage of the difference between the sources and is provably superior to oblivious RPs. Additionally, we provide extensive experiments comparing RPs with our approach showing significant performance lifts in fast matrix multiplication, regression and classification problems.