Dynamic Compositionality in Recursive Neural Networks with Structure-aware Tag Representations

TL;DR

This paper introduces a novel recursive neural network architecture that incorporates syntactic tags through a structure-aware tag representation, enhancing compositionality and improving performance on sentence-level NLP tasks.

Contribution

The authors propose a new RvNN model that integrates syntactic tags via a separate tree-LSTM, enabling dynamic compositionality based on both structure and tags.

Findings

Achieves superior or competitive results on sentiment analysis.

Outperforms previous tree-structured models.

Demonstrates the effectiveness of structure-aware tag representations.

Abstract

Most existing recursive neural network (RvNN) architectures utilize only the structure of parse trees, ignoring syntactic tags which are provided as by-products of parsing. We present a novel RvNN architecture that can provide dynamic compositionality by considering comprehensive syntactic information derived from both the structure and linguistic tags. Specifically, we introduce a structure-aware tag representation constructed by a separate tag-level tree-LSTM. With this, we can control the composition function of the existing word-level tree-LSTM by augmenting the representation as a supplementary input to the gate functions of the tree-LSTM. In extensive experiments, we show that models built upon the proposed architecture obtain superior or competitive performance on several sentence-level tasks such as sentiment analysis and natural language inference when compared against previous tree-structured models and other sophisticated neural models.