# Improving Tree-LSTM with Tree Attention

## TL;DR

This paper introduces a generalized attention mechanism for Tree-LSTM models to better capture tree structures in NLP, leading to improved performance on semantic relatedness tasks.

## Contribution

It proposes a novel attention framework integrated into Tree-LSTM for dependency and constituency trees, enhancing their ability to model hierarchical language structures.

## Key findings

- Achieved superior results on semantic relatedness compared to non-attention Tree-LSTM models.
- Outperformed other neural and non-neural methods without attention.
- Performed well against attention-based Tree-LSTM variants.

## Abstract

In Natural Language Processing (NLP), we often need to extract information from tree topology. Sentence structure can be represented via a dependency tree or a constituency tree structure. For this reason, a variant of LSTMs, named Tree-LSTM, was proposed to work on tree topology. In this paper, we design a generalized attention framework for both dependency and constituency trees by encoding variants of decomposable attention inside a Tree-LSTM cell. We evaluated our models on a semantic relatedness task and achieved notable results compared to Tree-LSTM based methods with no attention as well as other neural and non-neural methods and good results compared to Tree-LSTM based methods with attention.