Compact Random Feature Maps

TL;DR

This paper introduces CRAFTMaps, a new method for polynomial kernel approximation that is more compact and accurate, addressing deficiencies in previous approaches and enabling efficient learning of non-linear classifiers.

Contribution

The paper proposes CRAFTMaps, a novel compact random feature map for polynomial kernels, with proven error bounds and efficient algorithms for classifier training.

Findings

CRAFTMaps outperform previous kernel approximation methods in accuracy.

Structured random matrices enable efficient generation of CRAFTMaps.

Experiments show competitive performance on standard datasets.

Abstract

Kernel approximation using randomized feature maps has recently gained a lot of interest. In this work, we identify that previous approaches for polynomial kernel approximation create maps that are rank deficient, and therefore do not utilize the capacity of the projected feature space effectively. To address this challenge, we propose compact random feature maps (CRAFTMaps) to approximate polynomial kernels more concisely and accurately. We prove the error bounds of CRAFTMaps demonstrating their superior kernel reconstruction performance compared to the previous approximation schemes. We show how structured random matrices can be used to efficiently generate CRAFTMaps, and present a single-pass algorithm using CRAFTMaps to learn non-linear multi-class classifiers. We present experiments on multiple standard data-sets with performance competitive with state-of-the-art results.