A Neural Transducer
Navdeep Jaitly, David Sussillo, Quoc V. Le, Oriol Vinyals, Ilya, Sutskever, Samy Bengio

TL;DR
The paper introduces a Neural Transducer capable of making incremental predictions on streaming data, effectively handling long sequences and enabling real-time output without reprocessing entire inputs.
Contribution
It presents a novel neural transducer model that makes incremental predictions conditioned on partial inputs, trained with dynamic programming to handle discrete decision-making.
Findings
Performs well on streaming data tasks
Handles long sequences without attention mechanisms
Enables real-time incremental predictions
Abstract
Sequence-to-sequence models have achieved impressive results on various tasks. However, they are unsuitable for tasks that require incremental predictions to be made as more data arrives or tasks that have long input sequences and output sequences. This is because they generate an output sequence conditioned on an entire input sequence. In this paper, we present a Neural Transducer that can make incremental predictions as more input arrives, without redoing the entire computation. Unlike sequence-to-sequence models, the Neural Transducer computes the next-step distribution conditioned on the partially observed input sequence and the partially generated sequence. At each time step, the transducer can decide to emit zero to many output symbols. The data can be processed using an encoder and presented as input to the transducer. The discrete decision to emit a symbol at every time step…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsNeural Networks and Applications · Topic Modeling · Time Series Analysis and Forecasting
