Training with Exploration Improves a Greedy Stack-LSTM Parser

TL;DR

This paper enhances a greedy Stack-LSTM dependency parser by incorporating a training-with-exploration approach using dynamic oracles, leading to improved parsing accuracy for English and Chinese.

Contribution

It introduces a novel training method with exploration for neural parsers, replacing traditional cross-entropy training with dynamic oracle-based exploration.

Findings

Significant accuracy improvements on English and Chinese parsing tasks

Effective adaptation of training with exploration to neural network models

Demonstrates the benefits of dynamic oracle-based training in dependency parsing

Abstract

We adapt the greedy Stack-LSTM dependency parser of Dyer et al. (2015) to support a training-with-exploration procedure using dynamic oracles(Goldberg and Nivre, 2013) instead of cross-entropy minimization. This form of training, which accounts for model predictions at training time rather than assuming an error-free action history, improves parsing accuracies for both English and Chinese, obtaining very strong results for both languages. We discuss some modifications needed in order to get training with exploration to work well for a probabilistic neural-network.