New Tight Relaxations of Rank Minimization for Multi-Task Learning

TL;DR

This paper introduces two new formulations for multi-task learning that better approximate rank minimization, enabling more accurate shared subspace learning and outperforming existing methods.

Contribution

The paper presents two novel regularization terms that tightly approximate rank minimization and a re-weighted iterative strategy to solve the NP-hard problem.

Findings

Methods effectively recover shared low-rank structures.

Outperform existing multi-task learning approaches.

Demonstrated on benchmark datasets.

Abstract

Multi-task learning has been observed by many researchers, which supposes that different tasks can share a low-rank common yet latent subspace. It means learning multiple tasks jointly is better than learning them independently. In this paper, we propose two novel multi-task learning formulations based on two regularization terms, which can learn the optimal shared latent subspace by minimizing the exactly $k$ minimal singular values. The proposed regularization terms are the more tight approximations of rank minimization than trace norm. But it's an NP-hard problem to solve the exact rank minimization problem. Therefore, we design a novel re-weighted based iterative strategy to solve our models, which can tactically handle the exact rank minimization problem by setting a large penalizing parameter. Experimental results on benchmark datasets demonstrate that our methods can correctly recover the low-rank structure shared across tasks, and outperform related multi-task learning methods.