An improved neural network model for joint POS tagging and dependency parsing

TL;DR

This paper introduces an enhanced neural network model that jointly performs POS tagging and dependency parsing, achieving state-of-the-art results on multiple benchmarks and improving upon existing models in accuracy and downstream tasks.

Contribution

The paper presents a novel joint model combining POS tagging and dependency parsing using a BiLSTM-based component, extending the BIST parser for improved performance.

Findings

Achieves 94.51% UAS and 92.87% LAS on English Penn Treebank

Outperforms baseline UDPipe with 0.8% higher POS accuracy and 3.6% higher LAS on 61 UD treebanks

Sets new state-of-the-art results in biomedical event extraction and opinion analysis

Abstract

We propose a novel neural network model for joint part-of-speech (POS) tagging and dependency parsing. Our model extends the well-known BIST graph-based dependency parser (Kiperwasser and Goldberg, 2016) by incorporating a BiLSTM-based tagging component to produce automatically predicted POS tags for the parser. On the benchmark English Penn treebank, our model obtains strong UAS and LAS scores at 94.51% and 92.87%, respectively, producing 1.5+% absolute improvements to the BIST graph-based parser, and also obtaining a state-of-the-art POS tagging accuracy at 97.97%. Furthermore, experimental results on parsing 61 "big" Universal Dependencies treebanks from raw texts show that our model outperforms the baseline UDPipe (Straka and Strakov\'a, 2017) with 0.8% higher average POS tagging score and 3.6% higher average LAS score. In addition, with our model, we also obtain state-of-the-art downstream task scores for biomedical event extraction and opinion analysis applications. Our code is available together with all pre-trained models at: https://github.com/datquocnguyen/jPTDP