A Survey of Multi-task Learning in Natural Language Processing: Regarding Task Relatedness and Training Methods

TL;DR

This survey reviews recent multi-task learning methods in NLP, focusing on task relatedness and training strategies, and summarizes two main approaches: joint training and multi-step training, highlighting applications and future directions.

Contribution

It categorizes NLP multi-task learning methods based on task relatedness and provides a comprehensive overview of recent advances and future research directions.

Findings

Two main multi-task training methods identified: joint and multi-step training.

Summarizes task relationships across various NLP applications.

Discusses future challenges and directions in NLP multi-task learning.

Abstract

Multi-task learning (MTL) has become increasingly popular in natural language processing (NLP) because it improves the performance of related tasks by exploiting their commonalities and differences. Nevertheless, it is still not understood very well how multi-task learning can be implemented based on the relatedness of training tasks. In this survey, we review recent advances of multi-task learning methods in NLP, with the aim of summarizing them into two general multi-task training methods based on their task relatedness: (i) joint training and (ii) multi-step training. We present examples in various NLP downstream applications, summarize the task relationships and discuss future directions of this promising topic.