Geometry Aware Mappings for High Dimensional Sparse Factors

TL;DR

This paper introduces a geometry-aware mapping framework that leverages the structure of sparse vectors to accelerate high-dimensional matrix factorization computations with minimal accuracy loss.

Contribution

It proposes a novel geometry-aware permutation approach on a tessellated sphere to generate efficient sparse embeddings for latent factors.

Findings

Faster runtime performance in factorization tasks

Minimal accuracy degradation with the new embeddings

Effective exploitation of angular closeness in sparse vectors

Abstract

While matrix factorisation models are ubiquitous in large scale recommendation and search, real time application of such models requires inner product computations over an intractably large set of item factors. In this manuscript we present a novel framework that uses the inverted index representation to exploit structural properties of sparse vectors to significantly reduce the run time computational cost of factorisation models. We develop techniques that use geometry aware permutation maps on a tessellated unit sphere to obtain high dimensional sparse embeddings for latent factors with sparsity patterns related to angular closeness of the original latent factors. We also design several efficient and deterministic realisations within this framework and demonstrate with experiments that our techniques lead to faster run time operation with minimal loss of accuracy.