Multi-Task Learning with Deep Neural Networks: A Survey

TL;DR

This survey reviews deep neural network multi-task learning methods, discussing architectures, optimization, task relationships, and benchmarks, highlighting recent advances and ongoing challenges in the field.

Contribution

It provides a comprehensive overview of deep MTL techniques, categorizing methods and summarizing recent developments and benchmarks.

Findings

Categorization of deep MTL architectures, optimization, and task relationships

Summary of well-established and recent MTL methods

Overview of common multi-task benchmarks

Abstract

Multi-task learning (MTL) is a subfield of machine learning in which multiple tasks are simultaneously learned by a shared model. Such approaches offer advantages like improved data efficiency, reduced overfitting through shared representations, and fast learning by leveraging auxiliary information. However, the simultaneous learning of multiple tasks presents new design and optimization challenges, and choosing which tasks should be learned jointly is in itself a non-trivial problem. In this survey, we give an overview of multi-task learning methods for deep neural networks, with the aim of summarizing both the well-established and most recent directions within the field. Our discussion is structured according to a partition of the existing deep MTL techniques into three groups: architectures, optimization methods, and task relationship learning. We also provide a summary of common multi-task benchmarks.