Explicit Approximations of the Gaussian Kernel

TL;DR

This paper introduces Taylor features, a polynomial approximation of the Gaussian kernel using Taylor expansion, which offers a computationally efficient alternative to random Fourier features, especially for large datasets and online training.

Contribution

It presents a novel polynomial feature approximation of the Gaussian kernel that improves computational efficiency over existing methods like random Fourier features.

Findings

Taylor features provide better approximation with lower computational cost.

Suitable for large datasets and online or stochastic training.

Offers an alternative to random Fourier features for kernel approximation.

Abstract

We investigate training and using Gaussian kernel SVMs by approximating the kernel with an explicit finite- dimensional polynomial feature representation based on the Taylor expansion of the exponential. Although not as efficient as the recently-proposed random Fourier features [Rahimi and Recht, 2007] in terms of the number of features, we show how this polynomial representation can provide a better approximation in terms of the computational cost involved. This makes our "Taylor features" especially attractive for use on very large data sets, in conjunction with online or stochastic training.