Random Feature Maps for Dot Product Kernels

TL;DR

This paper introduces low distortion randomized feature maps for approximating dot product kernels, enabling efficient kernel methods by embedding into low-dimensional Euclidean spaces with high confidence.

Contribution

It extends existing kernel approximation techniques by providing explicit low-dimensional embeddings for all dot product kernels based on harmonic analysis.

Findings

High-confidence approximations of dot product kernels

Explicit low-dimensional Euclidean embeddings

Theoretical guarantees based on harmonic analysis

Abstract

Approximating non-linear kernels using feature maps has gained a lot of interest in recent years due to applications in reducing training and testing times of SVM classifiers and other kernel based learning algorithms. We extend this line of work and present low distortion embeddings for dot product kernels into linear Euclidean spaces. We base our results on a classical result in harmonic analysis characterizing all dot product kernels and use it to define randomized feature maps into explicit low dimensional Euclidean spaces in which the native dot product provides an approximation to the dot product kernel with high confidence.