Fast Learning Rate of lp-MKL and its Minimax Optimality

TL;DR

This paper establishes a sharp, minimax optimal generalization bound for lp-MKL, a generalized multiple kernel learning framework with lp-mixed-norm regularization, using localization techniques and eigenvalue decay rates.

Contribution

It introduces a new sharp generalization bound for lp-MKL with lp-mixed-norm regularization and proves its minimax optimality under certain conditions.

Findings

Derived a sharp learning rate based on eigenvalue decay

Proved the minimax optimality of the learning rate

Demonstrated the impact of kernel eigenvalue decay on convergence speed

Abstract

In this paper, we give a new sharp generalization bound of lp-MKL which is a generalized framework of multiple kernel learning (MKL) and imposes lp-mixed-norm regularization instead of l1-mixed-norm regularization. We utilize localization techniques to obtain the sharp learning rate. The bound is characterized by the decay rate of the eigenvalues of the associated kernels. A larger decay rate gives a faster convergence rate. Furthermore, we give the minimax learning rate on the ball characterized by lp-mixed-norm in the product space. Then we show that our derived learning rate of lp-MKL achieves the minimax optimal rate on the lp-mixed-norm ball.