Multi-task Sparse Structure Learning

TL;DR

This paper introduces a novel multi-task learning model that jointly estimates task relationships and parameters, leveraging sparse structure learning to enhance performance across regression, classification, and climate modeling tasks.

Contribution

The paper proposes a new family of models for multi-task learning that automatically learns task relationship structures using sparse Gaussian graphical models, applicable to various problems.

Findings

Effective on synthetic and benchmark datasets

Outperforms existing methods in climate model output combination

Improves generalization by learning task structures

Abstract

Multi-task learning (MTL) aims to improve generalization performance by learning multiple related tasks simultaneously. While sometimes the underlying task relationship structure is known, often the structure needs to be estimated from data at hand. In this paper, we present a novel family of models for MTL, applicable to regression and classification problems, capable of learning the structure of task relationships. In particular, we consider a joint estimation problem of the task relationship structure and the individual task parameters, which is solved using alternating minimization. The task relationship structure learning component builds on recent advances in structure learning of Gaussian graphical models based on sparse estimators of the precision (inverse covariance) matrix. We illustrate the effectiveness of the proposed model on a variety of synthetic and benchmark datasets for regression and classification. We also consider the problem of combining climate model outputs for better projections of future climate, with focus on temperature in South America, and show that the proposed model outperforms several existing methods for the problem.