Neural Search Space in Gboard Decoder

TL;DR

This paper introduces Neural Search Space, replacing N-gram language models with neural network models in Gboard decoding, leading to improved suggestion quality by incorporating long-range context.

Contribution

It proposes a novel method to integrate neural network language models into the Gboard decoder's search space dynamically at runtime, overcoming N-gram limitations.

Findings

Reduced Words Modified Ratio by up to 1.19% across locales

Improved suggestion quality with acceptable latency increase

Redesigned FST structure for neural network integration

Abstract

Gboard Decoder produces suggestions by looking for paths that best match input touch points on the context aware search space, which is backed by the language Finite State Transducers (FST). The language FST is currently an N-gram language model (LM). However, N-gram LMs, limited in context length, are known to have sparsity problem under device model size constraint. In this paper, we propose \textbf{Neural Search Space} which substitutes the N-gram LM with a Neural Network LM (NN-LM) and dynamically constructs the search space during decoding. Specifically, we integrate the long range context awareness of NN-LM into the search space by converting its outputs given context, into the language FST at runtime. This involves language FST structure redesign, pruning strategy tuning, and data structure optimizations. Online experiments demonstrate improved quality results, reducing Words Modified Ratio by [0.26\%, 1.19\%] on various locales with acceptable latency increases. This work opens new avenues for further improving keyboard decoding quality by enhancing neural LM more directly.