Neural Probabilistic Model for Non-projective MST Parsing

TL;DR

This paper introduces a neural probabilistic model for non-projective dependency parsing that leverages bi-directional LSTM-CNNs and Kirchhoff's Matrix-Tree Theorem to efficiently compute probabilities and achieve state-of-the-art results across multiple languages.

Contribution

It presents a novel neural probabilistic framework for non-projective MST parsing that combines structured probabilistic modeling with neural representations and efficient inference techniques.

Findings

Achieves state-of-the-art performance on nine datasets.

Efficient end-to-end training via back-propagation.

Effective across 14 languages and 17 datasets.

Abstract

In this paper, we propose a probabilistic parsing model, which defines a proper conditional probability distribution over non-projective dependency trees for a given sentence, using neural representations as inputs. The neural network architecture is based on bi-directional LSTM-CNNs which benefits from both word- and character-level representations automatically, by using combination of bidirectional LSTM and CNN. On top of the neural network, we introduce a probabilistic structured layer, defining a conditional log-linear model over non-projective trees. We evaluate our model on 17 different datasets, across 14 different languages. By exploiting Kirchhoff's Matrix-Tree Theorem (Tutte, 1984), the partition functions and marginals can be computed efficiently, leading to a straight-forward end-to-end model training procedure via back-propagation. Our parser achieves state-of-the-art parsing performance on nine datasets.