Semisoft Task Clustering for Multi-Task Learning

TL;DR

This paper introduces a semisoft task clustering method for multi-task learning that identifies task clusters, handles mixed tasks, and selects relevant features, improving performance on synthetic and real datasets.

Contribution

It proposes a novel semisoft clustering approach for MTL that reveals task structures and feature relevance, along with an efficient three-step optimization algorithm.

Findings

Effective in synthetic and real-world datasets

Handles pure and mixed tasks simultaneously

Extends to robust task clustering

Abstract

Multi-task learning (MTL) aims to improve the performance of multiple related prediction tasks by leveraging useful information from them. Due to their flexibility and ability to reduce unknown coefficients substantially, the task-clustering-based MTL approaches have attracted considerable attention. Motivated by the idea of semisoft clustering of data, we propose a semisoft task clustering approach, which can simultaneously reveal the task cluster structure for both pure and mixed tasks as well as select the relevant features. The main assumption behind our approach is that each cluster has some pure tasks, and each mixed task can be represented by a linear combination of pure tasks in different clusters. To solve the resulting non-convex constrained optimization problem, we design an efficient three-step algorithm. The experimental results based on synthetic and real-world datasets validate the effectiveness and efficiency of the proposed approach. Finally, we extend the proposed approach to a robust task clustering problem.