Compositional Distributional Semantics with Long Short Term Memory
Phong Le, Willem Zuidema

TL;DR
This paper introduces an LSTM-based extension to recursive neural networks, enabling better long-range dependency capture and addressing vanishing gradient issues, leading to improved sentiment analysis performance.
Contribution
It presents a novel LSTM-based recursive neural network architecture that enhances long-distance information retention in compositional semantics.
Findings
Outperformed traditional neural network composition on Stanford Sentiment Treebank
Successfully captured long-range dependencies in parse trees
Addressed vanishing gradient problem in recursive neural networks
Abstract
We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register (the `memory cell') and used much later higher up in the parse tree. This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies. Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
