Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders

TL;DR

This paper presents DIORA, an unsupervised deep autoencoder model that induces syntactic trees and learns representations for sentence constituents, achieving state-of-the-art results in unsupervised constituency parsing.

Contribution

It introduces a novel deep inside-outside recursive autoencoder that jointly learns sentence structure and representations without supervision.

Findings

Achieves new state-of-the-art F1 scores in unsupervised constituency parsing.

Effectively models sentence syntax using dynamic programming over all possible binary trees.

Demonstrates strong performance on WSJ and MultiNLI datasets.

Abstract

We introduce deep inside-outside recursive autoencoders (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree. Our approach predicts each word in an input sentence conditioned on the rest of the sentence and uses inside-outside dynamic programming to consider all possible binary trees over the sentence. At test time the CKY algorithm extracts the highest scoring parse. DIORA achieves a new state-of-the-art F1 in unsupervised binary constituency parsing (unlabeled) in two benchmark datasets, WSJ and MultiNLI.