When is multitask learning effective? Semantic sequence prediction under varying data conditions

TL;DR

This paper investigates the conditions under which multitask learning improves semantic sequence prediction, highlighting that its effectiveness depends on data characteristics and auxiliary task properties.

Contribution

It introduces a comprehensive evaluation of MTL for semantic tasks, including a novel auxiliary task setup, and identifies data-dependent factors influencing success.

Findings

MTL improves only 1 of 5 semantic tasks significantly.

Auxiliary tasks with compact, uniform label distributions are more effective.

Effectiveness of MTL varies based on data and task characteristics.

Abstract

Multitask learning has been applied successfully to a range of tasks, mostly morphosyntactic. However, little is known on when MTL works and whether there are data characteristics that help to determine its success. In this paper we evaluate a range of semantic sequence labeling tasks in a MTL setup. We examine different auxiliary tasks, amongst which a novel setup, and correlate their impact to data-dependent conditions. Our results show that MTL is not always effective, significant improvements are obtained only for 1 out of 5 tasks. When successful, auxiliary tasks with compact and more uniform label distributions are preferable.