Sparse coding for multitask and transfer learning

TL;DR

This paper explores using sparse coding and dictionary learning for multitask and transfer learning, demonstrating improved generalization and performance over existing methods through theoretical bounds and empirical experiments.

Contribution

It introduces a novel approach that models task parameters as sparse combinations of dictionary atoms, with theoretical error bounds and empirical validation.

Findings

Outperforms single task learning and previous dense methods

Provides theoretical bounds on generalization error

Shows advantages on synthetic and real datasets

Abstract

We investigate the use of sparse coding and dictionary learning in the context of multitask and transfer learning. The central assumption of our learning method is that the tasks parameters are well approximated by sparse linear combinations of the atoms of a dictionary on a high or infinite dimensional space. This assumption, together with the large quantity of available data in the multitask and transfer learning settings, allows a principled choice of the dictionary. We provide bounds on the generalization error of this approach, for both settings. Numerical experiments on one synthetic and two real datasets show the advantage of our method over single task learning, a previous method based on orthogonal and dense representation of the tasks and a related method learning task grouping.