Convex Multitask Learning with Flexible Task Clusters

TL;DR

This paper introduces a convex multitask learning approach that models task relationships at the feature level, allowing for flexible, feature-specific task clusters without predefining the number of clusters, leading to improved accuracy.

Contribution

It proposes a novel convex formulation for feature-level task clustering in multitask learning, which is computationally efficient and adaptable to various task relationships.

Findings

Consistently high accuracy across synthetic and real datasets.

Feature-specific task clusters align with known data structures.

Efficient optimization via accelerated proximal methods.

Abstract

Traditionally, multitask learning (MTL) assumes that all the tasks are related. This can lead to negative transfer when tasks are indeed incoherent. Recently, a number of approaches have been proposed that alleviate this problem by discovering the underlying task clusters or relationships. However, they are limited to modeling these relationships at the task level, which may be restrictive in some applications. In this paper, we propose a novel MTL formulation that captures task relationships at the feature-level. Depending on the interactions among tasks and features, the proposed method construct different task clusters for different features, without even the need of pre-specifying the number of clusters. Computationally, the proposed formulation is strongly convex, and can be efficiently solved by accelerated proximal methods. Experiments are performed on a number of synthetic and real-world data sets. Under various degrees of task relationships, the accuracy of the proposed method is consistently among the best. Moreover, the feature-specific task clusters obtained agree with the known/plausible task structures of the data.