Long Short-Term Memory-Networks for Machine Reading

TL;DR

This paper introduces an enhanced LSTM-based model with integrated memory networks and attention mechanisms for improved machine reading, demonstrating superior performance on language tasks.

Contribution

It proposes a novel LSTM architecture augmented with a memory network and neural attention, enabling better structured input handling and reasoning.

Findings

Outperforms state-of-the-art on language modeling

Achieves competitive results in sentiment analysis

Improves natural language inference accuracy

Abstract

In this paper we address the question of how to render sequence-level networks better at handling structured input. We propose a machine reading simulator which processes text incrementally from left to right and performs shallow reasoning with memory and attention. The reader extends the Long Short-Term Memory architecture with a memory network in place of a single memory cell. This enables adaptive memory usage during recurrence with neural attention, offering a way to weakly induce relations among tokens. The system is initially designed to process a single sequence but we also demonstrate how to integrate it with an encoder-decoder architecture. Experiments on language modeling, sentiment analysis, and natural language inference show that our model matches or outperforms the state of the art.