Schema-Free Dependency Parsing via Sequence Generation

TL;DR

This paper introduces a universal, schema-free dependency parsing method using sequence generation with pre-trained language models, capable of handling both syntactic and semantic parsing without auxiliary structures.

Contribution

The proposed DPSG method enables schema-free, multi-schemata dependency parsing solely with pre-trained language models, achieving state-of-the-art results on multiple benchmarks.

Findings

Achieves comparable results to top methods on DP benchmarks.

Attains state-of-the-art performance in CODT and SemEval16.

Demonstrates the potential of sequence generation for universal dependency parsing.

Abstract

Dependency parsing aims to extract syntactic dependency structure or semantic dependency structure for sentences. Existing methods suffer the drawbacks of lacking universality or highly relying on the auxiliary decoder. To remedy these drawbacks, we propose to achieve universal and schema-free Dependency Parsing (DP) via Sequence Generation (SG) DPSG by utilizing only the pre-trained language model (PLM) without any auxiliary structures or parsing algorithms. We first explore different serialization designing strategies for converting parsing structures into sequences. Then we design dependency units and concatenate these units into the sequence for DPSG. Thanks to the high flexibility of the sequence generation, our DPSG can achieve both syntactic DP and semantic DP using a single model. By concatenating the prefix to indicate the specific schema with the sequence, our DPSG can even accomplish multi-schemata parsing. The effectiveness of our DPSG is demonstrated by the experiments on widely used DP benchmarks, i.e., PTB, CODT, SDP15, and SemEval16. DPSG achieves comparable results with the first-tier methods on all the benchmarks and even the state-of-the-art (SOTA) performance in CODT and SemEval16. This paper demonstrates our DPSG has the potential to be a new parsing paradigm. We will release our codes upon acceptance.