Robust Task Clustering for Deep Many-Task Learning

TL;DR

This paper introduces a robust task clustering method for deep multi-task learning that uses matrix completion to accurately identify task groups, improving model training and adaptation for diverse tasks.

Contribution

It proposes a novel task clustering algorithm based on matrix completion, addressing asymmetry and unreliability in transfer performance matrices for better task partitioning.

Findings

Effective discovery of task clusters in sentiment and dialog classification.

Enhanced multi-task learning performance with flexible task grouping.

Empirical improvements in few-shot learning with multiple metrics.

Abstract

We investigate task clustering for deep-learning based multi-task and few-shot learning in a many-task setting. We propose a new method to measure task similarities with cross-task transfer performance matrix for the deep learning scenario. Although this matrix provides us critical information regarding similarity between tasks, its asymmetric property and unreliable performance scores can affect conventional clustering methods adversely. Additionally, the uncertain task-pairs, i.e., the ones with extremely asymmetric transfer scores, may collectively mislead clustering algorithms to output an inaccurate task-partition. To overcome these limitations, we propose a novel task-clustering algorithm by using the matrix completion technique. The proposed algorithm constructs a partially-observed similarity matrix based on the certainty of cluster membership of the task-pairs. We then use a matrix completion algorithm to complete the similarity matrix. Our theoretical analysis shows that under mild constraints, the proposed algorithm will perfectly recover the underlying "true" similarity matrix with a high probability. Our results show that the new task clustering method can discover task clusters for training flexible and superior neural network models in a multi-task learning setup for sentiment classification and dialog intent classification tasks. Our task clustering approach also extends metric-based few-shot learning methods to adapt multiple metrics, which demonstrates empirical advantages when the tasks are diverse.