Efficient Natural Language Response Suggestion for Smart Reply

TL;DR

This paper introduces a fast, neural network-based method for suggesting natural language responses in email applications, matching the quality of seq2seq models but with significantly reduced computational cost.

Contribution

The paper proposes an efficient neural network approach using n-gram embeddings and optimized search for real-time email response suggestions, outperforming traditional seq2seq models in speed.

Findings

Achieves comparable response quality to seq2seq models

Reduces computational requirements and latency

Successfully deployed in Gmail's Inbox application

Abstract

This paper presents a computationally efficient machine-learned method for natural language response suggestion. Feed-forward neural networks using n-gram embedding features encode messages into vectors which are optimized to give message-response pairs a high dot-product value. An optimized search finds response suggestions. The method is evaluated in a large-scale commercial e-mail application, Inbox by Gmail. Compared to a sequence-to-sequence approach, the new system achieves the same quality at a small fraction of the computational requirements and latency.