Complex-to-Real Sketches for Tensor Products with Applications to the Polynomial Kernel

TL;DR

This paper introduces a Complex-to-Real sketching method that reduces the embedding dimension for tensor product approximations, improving efficiency and accuracy in polynomial kernel applications.

Contribution

The authors propose a novel CtR sketch that lowers the embedding dimension from 3^p to 2^p, enhancing computational efficiency while maintaining real-valued outputs.

Findings

Achieves state-of-the-art accuracy in polynomial kernel approximation

Reduces embedding dimension complexity from 3^p to 2^p

Outperforms existing randomized sketching methods in speed and accuracy

Abstract

Randomized sketches of a tensor product of $p$ vectors follow a tradeoff between statistical efficiency and computational acceleration. Commonly used approaches avoid computing the high-dimensional tensor product explicitly, resulting in a suboptimal dependence of $\mathcal{O}(3^p)$ in the embedding dimension. We propose a simple Complex-to-Real (CtR) modification of well-known sketches that replaces real random projections by complex ones, incurring a lower $\mathcal{O}(2^p)$ factor in the embedding dimension. The output of our sketches is real-valued, which renders their downstream use straightforward. In particular, we apply our sketches to $p$-fold self-tensored inputs corresponding to the feature maps of the polynomial kernel. We show that our method achieves state-of-the-art performance in terms of accuracy and speed compared to other randomized approximations from the literature.