Semantic Role Labeling for Learner Chinese: the Importance of Syntactic Parsing and L2-L1 Parallel Data

TL;DR

This study investigates semantic role labeling for learner Chinese, highlighting the significance of syntactic parsing and parallel data, and introduces a new model that improves SRL accuracy on learner texts.

Contribution

It provides a new annotated dataset for learner Chinese SRL, evaluates existing systems on L2 data, and proposes a novel agreement-based model leveraging L2-L1 parallel data.

Findings

Parser-based systems are less affected by L2 data compared to L1-trained systems.

Syntactic parsing significantly impacts SRL performance on learner Chinese.

The proposed agreement-based model improves SRL F-score by 2.02 points.

Abstract

This paper studies semantic parsing for interlanguage (L2), taking semantic role labeling (SRL) as a case task and learner Chinese as a case language. We first manually annotate the semantic roles for a set of learner texts to derive a gold standard for automatic SRL. Based on the new data, we then evaluate three off-the-shelf SRL systems, i.e., the PCFGLA-parser-based, neural-parser-based and neural-syntax-agnostic systems, to gauge how successful SRL for learner Chinese can be. We find two non-obvious facts: 1) the L1-sentence-trained systems performs rather badly on the L2 data; 2) the performance drop from the L1 data to the L2 data of the two parser-based systems is much smaller, indicating the importance of syntactic parsing in SRL for interlanguages. Finally, the paper introduces a new agreement-based model to explore the semantic coherency information in the large-scale L2-L1 parallel data. We then show such information is very effective to enhance SRL for learner texts. Our model achieves an F-score of 72.06, which is a 2.02 point improvement over the best baseline.