Multi-Task Learning for Joint Semantic Role and Proto-Role Labeling

TL;DR

This paper introduces an end-to-end multi-task learning model that jointly predicts semantic roles and proto-roles without extra pre-training, achieving state-of-the-art results using static or contextual embeddings.

Contribution

It presents a novel multi-step architecture that jointly learns argument spans, syntactic heads, semantic roles, and proto-roles in an integrated manner without relying on additional pre-training or external annotations.

Findings

Achieves state-of-the-art proto-role prediction accuracy.

Effectively uses static and contextual embeddings without extra pre-training.

Joint learning improves overall semantic role and proto-role labeling performance.

Abstract

We put forward an end-to-end multi-step machine learning model which jointly labels semantic roles and the proto-roles of Dowty (1991), given a sentence and the predicates therein. Our best architecture first learns argument spans followed by learning the argument's syntactic heads. This information is shared with the next steps for predicting the semantic roles and proto-roles. We also experiment with transfer learning from argument and head prediction to role and proto-role labeling. We compare using static and contextual embeddings for words, arguments, and sentences. Unlike previous work, our model does not require pre-training or fine-tuning on additional tasks, beyond using off-the-shelf (static or contextual) embeddings and supervision. It also does not require argument spans, their semantic roles, and/or their gold syntactic heads as additional input, because it learns to predict all these during training. Our multi-task learning model raises the state-of-the-art predictions for most proto-roles.