Robust and Provable Guarantees for Sparse Random Embeddings

TL;DR

This paper provides explicit, sharper guarantees for sparse random embeddings, demonstrating significant empirical improvements over prior asymptotic bounds across diverse real-world datasets.

Contribution

It introduces explicit bounds with improved constants for sparse embeddings, enhancing theoretical guarantees and empirical performance over previous asymptotic results.

Findings

Bounds are explicit and sharper than previous asymptotic guarantees.

Empirical results show significant outperformance on real-world datasets.

Techniques include tighter estimates for quadratic chaos and bounds for sums of random variables.

Abstract

In this work, we improve upon the guarantees for sparse random embeddings, as they were recently provided and analyzed by Freksen at al. (NIPS'18) and Jagadeesan (NIPS'19). Specifically, we show that (a) our bounds are explicit as opposed to the asymptotic guarantees provided previously, and (b) our bounds are guaranteed to be sharper by practically significant constants across a wide range of parameters, including the dimensionality, sparsity and dispersion of the data. Moreover, we empirically demonstrate that our bounds significantly outperform prior works on a wide range of real-world datasets, such as collections of images, text documents represented as bags-of-words, and text sequences vectorized by neural embeddings. Behind our numerical improvements are techniques of broader interest, which improve upon key steps of previous analyses in terms of (c) tighter estimates for certain types of quadratic chaos, (d) establishing extreme properties of sparse linear forms, and (e) improvements on bounds for the estimation of sums of independent random variables.