# Learning Landmark-Based Ensembles with Random Fourier Features and Gradient Boosting

## TL;DR

This paper introduces a novel gradient boosting method that learns an ensemble of kernels using Random Fourier Features and barycenter optimization, offering improved flexibility and performance over existing kernel learning techniques.

## Contribution

It presents a new kernel learning algorithm that integrates RFF approximation with barycenter optimization within a gradient boosting framework, simplifying setup and enhancing results.

## Key findings

- Outperforms state-of-the-art kernel learning methods
- Demonstrates better generalization on benchmark datasets
- Shows increased versatility and ease of use

## Abstract

We propose a Gradient Boosting algorithm for learning an ensemble of kernel functions adapted to the task at hand. Unlike state-of-the-art Multiple Kernel Learning techniques that make use of a pre-computed dictionary of kernel functions to select from, at each iteration we fit a kernel by approximating it as a weighted sum of Random Fourier Features (RFF) and by optimizing their barycenter. This allows us to obtain a more versatile method, easier to setup and likely to have better performance. Our study builds on a recent result showing one can learn a kernel from RFF by computing the minimum of a PAC-Bayesian bound on the kernel alignment generalization loss, which is obtained efficiently from a closed-form solution. We conduct an experimental analysis to highlight the advantages of our method w.r.t. both Boosting-based and kernel-learning state-of-the-art methods.