Bidirectional Tree-Structured LSTM with Head Lexicalization

TL;DR

This paper introduces a bidirectional tree-structured LSTM with head lexicalization, enhancing tree representations by propagating head words and enabling top-down processing, leading to improved performance on sentiment analysis and question classification tasks.

Contribution

It proposes an automatic head-lexicalization method and a top-down bidirectional tree LSTM, which are novel extensions improving tree-structured neural models.

Findings

Achieved state-of-the-art results on Stanford Sentiment Treebank.

Demonstrated competitive performance on TREC question classification.

Both extensions improve the quality of tree representations.

Abstract

Sequential LSTM has been extended to model tree structures, giving competitive results for a number of tasks. Existing methods model constituent trees by bottom-up combinations of constituent nodes, making direct use of input word information only for leaf nodes. This is different from sequential LSTMs, which contain reference to input words for each node. In this paper, we propose a method for automatic head-lexicalization for tree-structure LSTMs, propagating head words from leaf nodes to every constituent node. In addition, enabled by head lexicalization, we build a tree LSTM in the top-down direction, which corresponds to bidirectional sequential LSTM structurally. Experiments show that both extensions give better representations of tree structures. Our final model gives the best results on the Standford Sentiment Treebank and highly competitive results on the TREC question type classification task.