Tensor Sketch: Fast and Scalable Polynomial Kernel Approximation

TL;DR

Tensor Sketch is a fast, scalable method for approximating polynomial kernels using random feature maps, enabling efficient kernel computations on large, high-dimensional datasets with theoretical error guarantees.

Contribution

Introduces Tensor Sketch, a novel efficient random feature map for polynomial kernel approximation with proven error bounds and scalable computation.

Findings

Computes low-dimensional embeddings in linear time relative to data size and feature dimensions.

Provides theoretical guarantees on approximation error.

Applicable to large-scale, high-dimensional datasets.

Abstract

Approximation of non-linear kernels using random feature maps has become a powerful technique for scaling kernel methods to large datasets. We propose $\textit{Tensor Sketch}$, an efficient random feature map for approximating polynomial kernels. Given $n$ training samples in $\mathbb{R}^d$ Tensor Sketch computes low-dimensional embeddings in $\mathbb{R}^D$ in time $\mathcal{O}\left( n(d+D \log{D}) \right)$ making it well-suited for high-dimensional and large-scale settings. We provide theoretical guarantees on the approximation error, ensuring the fidelity of the resulting kernel function estimates. We also discuss extensions and highlight applications where Tensor Sketch serves as a central computational tool.