Long Short-Term Memory Over Tree Structures

TL;DR

This paper introduces S-LSTM, an extension of LSTM to tree structures, enabling better modeling of hierarchical data like language and images, and demonstrates its effectiveness in natural language understanding tasks.

Contribution

The paper proposes S-LSTM, a novel recursive memory model for tree structures, improving hierarchical long-distance interaction modeling over previous recursive models.

Findings

S-LSTM outperforms state-of-the-art recursive models in semantic composition tasks.

Utilizing hierarchical structures improves performance in natural language understanding.

S-LSTM effectively models long-distance interactions in hierarchical data.

Abstract

The chain-structured long short-term memory (LSTM) has showed to be effective in a wide range of problems such as speech recognition and machine translation. In this paper, we propose to extend it to tree structures, in which a memory cell can reflect the history memories of multiple child cells or multiple descendant cells in a recursive process. We call the model S-LSTM, which provides a principled way of considering long-distance interaction over hierarchies, e.g., language or image parse structures. We leverage the models for semantic composition to understand the meaning of text, a fundamental problem in natural language understanding, and show that it outperforms a state-of-the-art recursive model by replacing its composition layers with the S-LSTM memory blocks. We also show that utilizing the given structures is helpful in achieving a performance better than that without considering the structures.