Counterfactual Language Model Adaptation for Suggesting Phrases

TL;DR

This paper introduces a counterfactual approach to adapt language models for suggesting phrases that writers are more likely to accept, improving text entry on mobile devices.

Contribution

It proposes a novel counterfactual training framework for phrase suggestion models, enabling offline optimization of suggestions based on acceptance likelihood.

Findings

Simple language models can effectively capture acceptability cues.

Counterfactual training improves suggestion acceptance rates.

The approach allows offline evaluation of interactive suggestion systems.

Abstract

Mobile devices use language models to suggest words and phrases for use in text entry. Traditional language models are based on contextual word frequency in a static corpus of text. However, certain types of phrases, when offered to writers as suggestions, may be systematically chosen more often than their frequency would predict. In this paper, we propose the task of generating suggestions that writers accept, a related but distinct task to making accurate predictions. Although this task is fundamentally interactive, we propose a counterfactual setting that permits offline training and evaluation. We find that even a simple language model can capture text characteristics that improve acceptability.