Dependency Parsing Across the Resource Spectrum: Evaluating Architectures on High and Low-Resource Languages

TL;DR

This paper compares different dependency parsers across high and low-resource languages, revealing that simpler models outperform transformers in low-resource settings, with transformers gaining an advantage as data increases.

Contribution

It provides a comprehensive evaluation of parser architectures on diverse languages, highlighting the conditions under which each performs best, especially in low-resource scenarios.

Findings

Biaffine LSTM outperforms transformers in low-resource regimes.

Transformers' advantage increases with more training data.

Morphological complexity affects transformer performance relative to data size.

Abstract

Transformer-based models achieve state-of-the-art dependency parsing for high-resource languages, yet their advantage over simpler architectures in low-resource settings remains poorly understood. We evaluate four parsers -- the Biaffine LSTM, Stack-Pointer Network, AfroXLMR-large, and RemBERT -- across ten typologically diverse languages, with a focus on low-resource African languages. We find that the Biaffine LSTM consistently outperforms transformer models in low-resource regimes, with transformers recovering their advantage as training data increases. The crossover falls within a resource range typical of treebanks for under-resourced languages. Morphological complexity (measured via MATTR) emerges as a significant secondary predictor of transformers' relative disadvantage after controlling for corpus size. These results indicate that the Biaffine LSTM may be better suited for syntactic tool development in low-resource regimes until sufficient annotated data is available to leverage the representational capacity of pre-trained transformers.