Multi-task Learning in Vector-valued Reproducing Kernel Banach Spaces with the $\ell^1$ Norm

TL;DR

This paper develops a new vector-valued reproducing kernel Banach space framework with an  norm for sparse multi-task learning, providing theoretical properties and demonstrating practical benefits through experiments.

Contribution

It introduces a novel class of vector-valued RKBS with  norm and defines multi-task admissible kernels, establishing a linear representer theorem for sparse multi-task learning.

Findings

The proposed kernels satisfy desirable properties including bounded Lebesgue constants.

The new regularization models outperform existing methods on synthetic and real-world data.

Numerical experiments validate the effectiveness of the  norm-based vector-valued RKBS.

Abstract

Targeting at sparse multi-task learning, we consider regularization models with an $\ell^1$ penalty on the coefficients of kernel functions. In order to provide a kernel method for this model, we construct a class of vector-valued reproducing kernel Banach spaces with the $\ell^1$ norm. The notion of multi-task admissible kernels is proposed so that the constructed spaces could have desirable properties including the crucial linear representer theorem. Such kernels are related to bounded Lebesgue constants of a kernel interpolation question. We study the Lebesgue constant of multi-task kernels and provide examples of admissible kernels. Furthermore, we present numerical experiments for both synthetic data and real-world benchmark data to demonstrate the advantages of the proposed construction and regularization models.