Multitask Learning via Shared Features: Algorithms and Hardness

TL;DR

This paper presents a polynomial-time multitask learning algorithm for Boolean functions over hypercubes using shared features, and demonstrates a computational separation showing limits of multitask learning efficiency.

Contribution

It introduces a new efficient algorithm for multitask learning of halfspaces with shared features and proves a separation showing some classes cannot be learned efficiently in this setting.

Findings

Polynomial-time algorithm for halfspaces with margin g

Sample complexity bounds for multitask learning

Existence of concept classes that cannot be efficiently multitask learned

Abstract

We investigate the computational efficiency of multitask learning of Boolean functions over the $d$-dimensional hypercube, that are related by means of a feature representation of size $k \ll d$ shared across all tasks. We present a polynomial time multitask learning algorithm for the concept class of halfspaces with margin $\gamma$, which is based on a simultaneous boosting technique and requires only $\textrm{poly}(k/\gamma)$ samples-per-task and $\textrm{poly}(k\log(d)/\gamma)$ samples in total. In addition, we prove a computational separation, showing that assuming there exists a concept class that cannot be learned in the attribute-efficient model, we can construct another concept class such that can be learned in the attribute-efficient model, but cannot be multitask learned efficiently -- multitask learning this concept class either requires super-polynomial time complexity or a much larger total number of samples.