Compositional Distributional Semantics with Long Short Term Memory

TL;DR

This paper introduces an LSTM-based extension to recursive neural networks, enabling better long-range dependency capture and addressing vanishing gradient issues, leading to improved sentiment analysis performance.

Contribution

It presents a novel LSTM-based recursive neural network architecture that enhances long-distance information retention in compositional semantics.

Findings

Outperformed traditional neural network composition on Stanford Sentiment Treebank

Successfully captured long-range dependencies in parse trees

Addressed vanishing gradient problem in recursive neural networks

Abstract

We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register (the `memory cell') and used much later higher up in the parse tree. This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies. Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.