Semi-supervised Autoencoding Projective Dependency Parsing

TL;DR

This paper introduces two semi-supervised autoencoding models for projective dependency parsing that leverage unlabeled data to improve parsing accuracy, outperforming previous semi-supervised approaches.

Contribution

It proposes two novel autoencoding models, LAP and GAP, for semi-supervised dependency parsing, with exact inference and shared parameters for labeled and unlabeled data.

Findings

Models improve performance with limited labeled data.

Outperform previous semi-supervised models.

Effective use of unlabeled data in dependency parsing.

Abstract

We describe two end-to-end autoencoding models for semi-supervised graph-based projective dependency parsing. The first model is a Locally Autoencoding Parser (LAP) encoding the input using continuous latent variables in a sequential manner; The second model is a Globally Autoencoding Parser (GAP) encoding the input into dependency trees as latent variables, with exact inference. Both models consist of two parts: an encoder enhanced by deep neural networks (DNN) that can utilize the contextual information to encode the input into latent variables, and a decoder which is a generative model able to reconstruct the input. Both LAP and GAP admit a unified structure with different loss functions for labeled and unlabeled data with shared parameters. We conducted experiments on WSJ and UD dependency parsing data sets, showing that our models can exploit the unlabeled data to improve the performance given a limited amount of labeled data, and outperform a previously proposed semi-supervised model.