Multi-task and Lifelong Learning of Kernels

TL;DR

This paper develops theoretical bounds for multi-task and lifelong kernel learning in SVMs, showing that learning multiple related tasks can reduce complexity and improve generalization, especially with many tasks and suitable kernels.

Contribution

It provides the first generalization bounds for multi-task and lifelong kernel learning, demonstrating benefits over single-task learning under mild conditions.

Findings

Learning multiple related tasks reduces generalization error.

As the number of tasks increases, the complexity approaches that of using an optimal kernel.

The benefits are significant when a suitable kernel exists within the family.

Abstract

We consider a problem of learning kernels for use in SVM classification in the multi-task and lifelong scenarios and provide generalization bounds on the error of a large margin classifier. Our results show that, under mild conditions on the family of kernels used for learning, solving several related tasks simultaneously is beneficial over single task learning. In particular, as the number of observed tasks grows, assuming that in the considered family of kernels there exists one that yields low approximation error on all tasks, the overhead associated with learning such a kernel vanishes and the complexity converges to that of learning when this good kernel is given to the learner.