Towards Better UD Parsing: Deep Contextualized Word Embeddings, Ensemble, and Treebank Concatenation

TL;DR

This paper presents an improved multilingual dependency parser that leverages deep contextualized embeddings, ensembling, and treebank concatenation, achieving top performance in the CoNLL 2018 shared task.

Contribution

It introduces the integration of deep contextualized word embeddings and ensembling techniques into a multilingual parser, along with methods for effective treebank concatenation.

Findings

Achieved highest LAS score of 75.84% in the shared task

Deep contextualized embeddings improved parsing accuracy

Ensembling and treebank concatenation contributed to performance gains

Abstract

This paper describes our system (HIT-SCIR) submitted to the CoNLL 2018 shared task on Multilingual Parsing from Raw Text to Universal Dependencies. We base our submission on Stanford's winning system for the CoNLL 2017 shared task and make two effective extensions: 1) incorporating deep contextualized word embeddings into both the part of speech tagger and parser; 2) ensembling parsers trained with different initialization. We also explore different ways of concatenating treebanks for further improvements. Experimental results on the development data show the effectiveness of our methods. In the final evaluation, our system was ranked first according to LAS (75.84%) and outperformed the other systems by a large margin.