Conditional Random Field Autoencoders for Unsupervised Structured Prediction

TL;DR

This paper presents a novel unsupervised learning framework using conditional random field autoencoders for structured prediction, enabling efficient training with rich features and demonstrating competitive results in NLP tasks.

Contribution

It introduces a new autoencoder framework for unsupervised structured prediction with overlapping features, connecting to existing methods and improving training efficiency.

Findings

Competitive performance on POS induction and word alignment.

Efficient training compared to feature-rich baselines.

Insightful connections to traditional autoencoders and multi-view learning.

Abstract

We introduce a framework for unsupervised learning of structured predictors with overlapping, global features. Each input's latent representation is predicted conditional on the observable data using a feature-rich conditional random field. Then a reconstruction of the input is (re)generated, conditional on the latent structure, using models for which maximum likelihood estimation has a closed-form. Our autoencoder formulation enables efficient learning without making unrealistic independence assumptions or restricting the kinds of features that can be used. We illustrate insightful connections to traditional autoencoders, posterior regularization and multi-view learning. We show competitive results with instantiations of the model for two canonical NLP tasks: part-of-speech induction and bitext word alignment, and show that training our model can be substantially more efficient than comparable feature-rich baselines.