Variable Selection and Task Grouping for Multi-Task Learning

TL;DR

This paper introduces a novel multi-task learning approach that uses low-rank matrix factorization with sparsity to perform variable selection and task grouping, improving generalization.

Contribution

It proposes a bi-convex optimization framework with sparsity constraints for variable selection and task grouping, validated through theoretical bounds and experiments.

Findings

Effective variable selection and task grouping demonstrated on datasets.

The method outperforms existing approaches in generalization performance.

Theoretical performance bounds are established.

Abstract

We consider multi-task learning, which simultaneously learns related prediction tasks, to improve generalization performance. We factorize a coefficient matrix as the product of two matrices based on a low-rank assumption. These matrices have sparsities to simultaneously perform variable selection and learn and overlapping group structure among the tasks. The resulting bi-convex objective function is minimized by alternating optimization where sub-problems are solved using alternating direction method of multipliers and accelerated proximal gradient descent. Moreover, we provide the performance bound of the proposed method. The effectiveness of the proposed method is validated for both synthetic and real-world datasets.