Not-So-Random Features

TL;DR

This paper introduces a Fourier-based kernel learning method that iteratively refines feature maps, offering theoretical guarantees and demonstrating improved performance over existing random features techniques.

Contribution

It presents a novel Fourier-analytic approach for kernel learning with rigorous guarantees and an iterative feature refinement process.

Findings

Scalable method with improved accuracy on synthetic datasets

Consistent performance gains over random features methods

Theoretical guarantees for optimality and generalization

Abstract

We propose a principled method for kernel learning, which relies on a Fourier-analytic characterization of translation-invariant or rotation-invariant kernels. Our method produces a sequence of feature maps, iteratively refining the SVM margin. We provide rigorous guarantees for optimality and generalization, interpreting our algorithm as online equilibrium-finding dynamics in a certain two-player min-max game. Evaluations on synthetic and real-world datasets demonstrate scalability and consistent improvements over related random features-based methods.