Generating Sequences With Recurrent Neural Networks

TL;DR

This paper demonstrates how LSTM recurrent neural networks can generate complex, realistic sequences such as text and handwriting, including style-conditioned handwriting synthesis, by predicting data points sequentially.

Contribution

It introduces a method for sequence generation with LSTMs that handles both discrete and real-valued data, extending to style-conditioned handwriting synthesis.

Findings

LSTM networks can generate complex sequences with long-range dependencies.

The approach produces highly realistic cursive handwriting.

Conditional generation enables style variation in handwriting synthesis.

Abstract

This paper shows how Long Short-term Memory recurrent neural networks can be used to generate complex sequences with long-range structure, simply by predicting one data point at a time. The approach is demonstrated for text (where the data are discrete) and online handwriting (where the data are real-valued). It is then extended to handwriting synthesis by allowing the network to condition its predictions on a text sequence. The resulting system is able to generate highly realistic cursive handwriting in a wide variety of styles.