A New Algorithm for Tessellated Kernel Learning

TL;DR

This paper introduces a scalable algorithm for tessellated kernel learning that extends to regression and outperforms neural networks and MKL on benchmarks, addressing previous limitations in kernel optimization.

Contribution

The paper presents a new 2-step algorithm for tessellated kernel learning that scales to large datasets and applies to regression, improving over prior SDP-based methods.

Findings

Scales to 10,000 data points

Extends to regression tasks

Outperforms neural nets and MKL on benchmarks

Abstract

The accuracy and complexity of machine learning algorithms based on kernel optimization are limited by the set of kernels over which they are able to optimize. An ideal set of kernels should: admit a linear parameterization (for tractability); be dense in the set of all kernels (for robustness); be universal (for accuracy). The recently proposed Tesselated Kernels (TKs) is currently the only known class which meets all three criteria. However, previous algorithms for optimizing TKs were limited to classification and relied on Semidefinite Programming (SDP) - limiting them to relatively small datasets. By contrast, the 2-step algorithm proposed here scales to 10,000 data points and extends to the regression problem. Furthermore, when applied to benchmark data, the algorithm demonstrates significant improvement in performance over Neural Nets and SimpleMKL with similar computation time.