An Exact Finite-dimensional Explicit Feature Map for Kernel Functions

TL;DR

This paper presents a method to construct explicit, finite-dimensional feature maps for arbitrary kernel functions, enabling primal-form kernel algorithms and simplifying their implementation.

Contribution

It introduces a universal explicit feature map for any kernel, allowing direct primal formulation of kernel methods like PCA and t-SNE.

Findings

Explicit feature maps enable primal kernel algorithms.

Application to PCA demonstrates direct derivation of kernelized algorithms.

Feature space visualization with t-SNE remains unchanged.

Abstract

Kernel methods in machine learning use a kernel function that takes two data points as input and returns their inner product after mapping them to a Hilbert space, implicitly and without actually computing the mapping. For many kernel functions, such as Gaussian and Laplacian kernels, the feature space is known to be infinite-dimensional, making operations in this space possible only implicitly. This implicit nature necessitates algorithms to be expressed using dual representations and the kernel trick. In this paper, given an arbitrary kernel function, we introduce an explicit, finite-dimensional feature map for any arbitrary kernel function that ensures the inner product of data points in the feature space equals the kernel function value, during both training and testing. The existence of this explicit mapping allows for kernelized algorithms to be formulated in their primal form, without the need for the kernel trick or the dual representation. As a first application, we demonstrate how to derive kernelized machine learning algorithms directly, without resorting to the dual representation, and apply this method specifically to PCA. As another application, without any changes to the t-SNE algorithm and its implementation, we use it for visualizing the feature space of kernel functions.