The DCU-EPFL Enhanced Dependency Parser at the IWPT 2021 Shared Task

TL;DR

This paper presents a dependency parser for Enhanced Universal Dependencies, utilizing transformer-based representations and post-processing, achieving competitive results in the IWPT 2021 Shared Task.

Contribution

The authors introduce a novel pipeline combining XLM-RoBERTa and post-processing techniques for parsing Enhanced UD graphs, with improvements from additional experiments.

Findings

Initial ELAS of 83.57 achieved

Enhanced system reaches ELAS of 88.04

Post-deadline modifications improve performance

Abstract

We describe the DCU-EPFL submission to the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies. The task involves parsing Enhanced UD graphs, which are an extension of the basic dependency trees designed to be more facilitative towards representing semantic structure. Evaluation is carried out on 29 treebanks in 17 languages and participants are required to parse the data from each language starting from raw strings. Our approach uses the Stanza pipeline to preprocess the text files, XLMRoBERTa to obtain contextualized token representations, and an edge-scoring and labeling model to predict the enhanced graph. Finally, we run a post-processing script to ensure all of our outputs are valid Enhanced UD graphs. Our system places 6th out of 9 participants with a coarse Enhanced Labeled Attachment Score (ELAS) of 83.57. We carry out additional post-deadline experiments which include using Trankit for pre-processing, XLM-RoBERTa-LARGE, treebank concatenation, and multitask learning between a basic and an enhanced dependency parser. All of these modifications improve our initial score and our final system has a coarse ELAS of 88.04.