Grid Long Short-Term Memory
Nal Kalchbrenner, Ivo Danihelka, Alex Graves

TL;DR
This paper presents Grid LSTM, a multidimensional LSTM architecture that enhances deep and sequential data processing, achieving state-of-the-art results in character prediction and translation tasks.
Contribution
The introduction of Grid LSTM, a multidimensional LSTM network connecting cells across layers and dimensions, unifying deep and sequential computation.
Findings
Achieves 1.47 bits per character on Wikipedia benchmark
Outperforms standard LSTM on algorithmic tasks
Outperforms phrase-based system in Chinese-English translation
Abstract
This paper introduces Grid Long Short-Term Memory, a network of LSTM cells arranged in a multidimensional grid that can be applied to vectors, sequences or higher dimensional data such as images. The network differs from existing deep LSTM architectures in that the cells are connected between network layers as well as along the spatiotemporal dimensions of the data. The network provides a unified way of using LSTM for both deep and sequential computation. We apply the model to algorithmic tasks such as 15-digit integer addition and sequence memorization, where it is able to significantly outperform the standard LSTM. We then give results for two empirical tasks. We find that 2D Grid LSTM achieves 1.47 bits per character on the Wikipedia character prediction benchmark, which is state-of-the-art among neural approaches. In addition, we use the Grid LSTM to define a novel two-dimensional…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTopic Modeling · Natural Language Processing Techniques · Multimodal Machine Learning Applications
MethodsSigmoid Activation · Tanh Activation · Long Short-Term Memory
