CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languages

TL;DR

This paper introduces a contrastive self-supervised learning approach to improve dependency parsing robustness in morphologically rich languages with free word order, achieving significant accuracy gains.

Contribution

It proposes a novel contrastive self-supervised method and modifications to dependency parsing models to enhance their robustness to word order variations in low-resource languages.

Findings

Average gain of 3.03/2.95 UAS/LAS points across 7 languages

Model robustness improved without position encoding

Effective data augmentation enhances parsing performance

Abstract

Neural dependency parsing has achieved remarkable performance for low resource morphologically rich languages. It has also been well-studied that morphologically rich languages exhibit relatively free word order. This prompts a fundamental investigation: Is there a way to enhance dependency parsing performance, making the model robust to word order variations utilizing the relatively free word order nature of morphologically rich languages? In this work, we examine the robustness of graph-based parsing architectures on 7 relatively free word order languages. We focus on scrutinizing essential modifications such as data augmentation and the removal of position encoding required to adapt these architectures accordingly. To this end, we propose a contrastive self-supervised learning method to make the model robust to word order variations. Furthermore, our proposed modification demonstrates a substantial average gain of 3.03/2.95 points in 7 relatively free word order languages, as measured by the UAS/LAS Score metric when compared to the best performing baseline.