Discriminative Neural Sentence Modeling by Tree-Based Convolution

TL;DR

This paper introduces a tree-based convolutional neural network that leverages sentence structure for improved discriminative sentence modeling, achieving state-of-the-art results in sentiment analysis and question classification.

Contribution

It presents a novel TBCNN architecture that effectively captures structural features from constituency and dependency trees for sentence classification.

Findings

TBCNN outperforms previous neural models on sentiment analysis.

TBCNN achieves superior results on question classification.

Visualization sheds light on the model's structural feature learning.

Abstract

This paper proposes a tree-based convolutional neural network (TBCNN) for discriminative sentence modeling. Our models leverage either constituency trees or dependency trees of sentences. The tree-based convolution process extracts sentences' structural features, and these features are aggregated by max pooling. Such architecture allows short propagation paths between the output layer and underlying feature detectors, which enables effective structural feature learning and extraction. We evaluate our models on two tasks: sentiment analysis and question classification. In both experiments, TBCNN outperforms previous state-of-the-art results, including existing neural networks and dedicated feature/rule engineering. We also make efforts to visualize the tree-based convolution process, shedding light on how our models work.