PCA-based Multi Task Learning: a Random Matrix Approach

TL;DR

This paper introduces a computationally efficient multi-task learning extension of PCA-based supervised methods, addressing negative transfer issues and demonstrating comparable performance to state-of-the-art techniques with lower computational costs.

Contribution

It presents a novel PCA-based multi-task learning approach with theoretical analysis and simple label-based counter-measures to prevent negative transfer.

Findings

Addresses negative transfer in multi-task learning

Achieves comparable performance with reduced computational cost

Supports findings with synthetic and real data experiments

Abstract

The article proposes and theoretically analyses a \emph{computationally efficient} multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes \cite{barshan2011supervised,bair2006prediction}. The analysis reveals that (i) by default learning may dramatically fail by suffering from \emph{negative transfer}, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks show that the proposed method achieves comparable performance with state-of-the-art MTL methods but at a \emph{significantly reduced computational cost}.