Gmail Smart Compose: Real-Time Assisted Writing

TL;DR

Smart Compose is a real-time, AI-powered email suggestion system integrated into Gmail, utilizing advanced neural language models and infrastructure to assist users in composing emails efficiently.

Contribution

The paper introduces a large-scale neural language model and novel deployment infrastructure for real-time email suggestions in Gmail.

Findings

High-quality suggestion prediction achieved

Effective real-time inference infrastructure developed

System successfully deployed in Gmail

Abstract

In this paper, we present Smart Compose, a novel system for generating interactive, real-time suggestions in Gmail that assists users in writing mails by reducing repetitive typing. In the design and deployment of such a large-scale and complicated system, we faced several challenges including model selection, performance evaluation, serving and other practical issues. At the core of Smart Compose is a large-scale neural language model. We leveraged state-of-the-art machine learning techniques for language model training which enabled high-quality suggestion prediction, and constructed novel serving infrastructure for high-throughput and real-time inference. Experimental results show the effectiveness of our proposed system design and deployment approach. This system is currently being served in Gmail.