Towards Semi-Supervised Learning for Deep Semantic Role Labeling

TL;DR

This paper introduces a semi-supervised approach for deep semantic role labeling that leverages syntactic constraints and unlabeled data, significantly improving performance in low-resource settings.

Contribution

It proposes a novel semi-supervised training method with syntactic-inconsistency loss, enhancing SRL accuracy with limited labeled data.

Findings

Achieves +1.58 F1 with 1% labeled data

Improves +0.78 F1 with 10% labeled data

Enhances inference F1 by +3.67 and +2.1 with 1% and 10% data

Abstract

Neural models have shown several state-of-the-art performances on Semantic Role Labeling (SRL). However, the neural models require an immense amount of semantic-role corpora and are thus not well suited for low-resource languages or domains. The paper proposes a semi-supervised semantic role labeling method that outperforms the state-of-the-art in limited SRL training corpora. The method is based on explicitly enforcing syntactic constraints by augmenting the training objective with a syntactic-inconsistency loss component and uses SRL-unlabeled instances to train a joint-objective LSTM. On CoNLL-2012 English section, the proposed semi-supervised training with 1%, 10% SRL-labeled data and varying amounts of SRL-unlabeled data achieves +1.58, +0.78 F1, respectively, over the pre-trained models that were trained on SOTA architecture with ELMo on the same SRL-labeled data. Additionally, by using the syntactic-inconsistency loss on inference time, the proposed model achieves +3.67, +2.1 F1 over pre-trained model on 1%, 10% SRL-labeled data, respectively.