On the Expressive Power of Kernel Methods and the Efficiency of Kernel Learning by Association Schemes

TL;DR

This paper explores the expressive capabilities of Euclidean kernels, a broad class including polynomial and RBF kernels, and presents structural insights and algorithms for kernel learning efficiency.

Contribution

It introduces Euclidean kernels, analyzes their structure, and provides limitations and efficient algorithms for learning kernels over various domains.

Findings

Structural characterization of Euclidean kernels

Limitations on their expressive power

Efficient algorithms for kernel learning

Abstract

We study the expressive power of kernel methods and the algorithmic feasibility of multiple kernel learning for a special rich class of kernels. Specifically, we define \emph{Euclidean kernels}, a diverse class that includes most, if not all, families of kernels studied in literature such as polynomial kernels and radial basis functions. We then describe the geometric and spectral structure of this family of kernels over the hypercube (and to some extent for any compact domain). Our structural results allow us to prove meaningful limitations on the expressive power of the class as well as derive several efficient algorithms for learning kernels over different domains.