The Benefit of Multitask Representation Learning

TL;DR

This paper explores a general framework for multitask representation learning, providing theoretical insights into when it outperforms independent task learning, especially in linear and kernel-based models.

Contribution

It introduces a unified method for multitask representation learning with theoretical conditions for its advantages over independent learning.

Findings

Conditions under which multitask learning outperforms independent learning

Regimes where multitask representation learning is beneficial based on sample size and task complexity

Applicability to linear models, kernel spaces, and deep networks

Abstract

We discuss a general method to learn data representations from multiple tasks. We provide a justification for this method in both settings of multitask learning and learning-to-learn. The method is illustrated in detail in the special case of linear feature learning. Conditions on the theoretical advantage offered by multitask representation learning over independent task learning are established. In particular, focusing on the important example of half-space learning, we derive the regime in which multitask representation learning is beneficial over independent task learning, as a function of the sample size, the number of tasks and the intrinsic data dimensionality. Other potential applications of our results include multitask feature learning in reproducing kernel Hilbert spaces and multilayer, deep networks.