Sparsity in multiple kernel learning

TL;DR

This paper investigates sparse multiple kernel learning using penalized empirical risk minimization, focusing on adaptivity to sparsity and unknown distributions, with theoretical guarantees on excess risk.

Contribution

It introduces a data-driven regularization approach for sparse multiple kernel learning and establishes oracle inequalities for its excess risk.

Findings

Method adapts to unknown distribution and sparsity

Provides oracle inequalities for excess risk

Effective in high-dimensional kernel settings

Abstract

The problem of multiple kernel learning based on penalized empirical risk minimization is discussed. The complexity penalty is determined jointly by the empirical $L_2$ norms and the reproducing kernel Hilbert space (RKHS) norms induced by the kernels with a data-driven choice of regularization parameters. The main focus is on the case when the total number of kernels is large, but only a relatively small number of them is needed to represent the target function, so that the problem is sparse. The goal is to establish oracle inequalities for the excess risk of the resulting prediction rule showing that the method is adaptive both to the unknown design distribution and to the sparsity of the problem.