A Convolutional Neural Network for Modelling Sentences

TL;DR

This paper introduces a convolutional neural network architecture called DCNN for sentence modeling, capable of capturing complex relations without parse trees, and demonstrates its effectiveness across various language understanding tasks.

Contribution

The paper presents the Dynamic Convolutional Neural Network (DCNN) with Dynamic k-Max Pooling, enabling flexible sentence representation without reliance on parse trees.

Findings

Achieves state-of-the-art results in sentiment and question classification

Reduces error by over 25% in Twitter sentiment prediction

Handles variable-length input effectively

Abstract

The ability to accurately represent sentences is central to language understanding. We describe a convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) that we adopt for the semantic modelling of sentences. The network uses Dynamic k-Max Pooling, a global pooling operation over linear sequences. The network handles input sentences of varying length and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations. The network does not rely on a parse tree and is easily applicable to any language. We test the DCNN in four experiments: small scale binary and multi-class sentiment prediction, six-way question classification and Twitter sentiment prediction by distant supervision. The network achieves excellent performance in the first three tasks and a greater than 25% error reduction in the last task with respect to the strongest baseline.