Empirical Evaluation of Multi-task Learning in Deep Neural Networks for Natural Language Processing

TL;DR

This paper systematically evaluates various multi-task learning architectures and mechanisms in NLP, aiming to understand their strengths and weaknesses and to develop improved hybrid models.

Contribution

It provides a comprehensive comparison of existing MTL methods in NLP and proposes new hybrid architectures to enhance performance.

Findings

Thorough comparison of MTL architectures across NLP tasks

Identification of strengths and weaknesses of existing methods

Development of hybrid models combining best features

Abstract

Multi-Task Learning (MTL) aims at boosting the overall performance of each individual task by leveraging useful information contained in multiple related tasks. It has shown great success in natural language processing (NLP). Currently, a number of MLT architectures and learning mechanisms have been proposed for various NLP tasks. However, there is no systematic exploration and comparison of different MLT architectures and learning mechanisms for their strong performance in-depth. In this paper, we conduct a thorough examination of typical MTL methods on a broad range of representative NLP tasks. Our primary goal is to understand the merits and demerits of existing MTL methods in NLP tasks, thus devising new hybrid architectures intended to combine their strengths.